You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "oleg-vlsk (via GitHub)" <gi...@apache.org> on 2025/05/22 02:28:11 UTC
[PR] IGNITE-25449 Add memory leak detection in LongRunningQueryTest for He… [ignite]
oleg-vlsk opened a new pull request, #12087:
URL: https://github.com/apache/ignite/pull/12087
…avyQueriesTracker
Thank you for submitting the pull request to the Apache Ignite.
In order to streamline the review of the contribution
we ask you to ensure the following steps have been taken:
### The Contribution Checklist
- [ ] There is a single JIRA ticket related to the pull request.
- [ ] The web-link to the pull request is attached to the JIRA ticket.
- [ ] The JIRA ticket has the _Patch Available_ state.
- [ ] The pull request body describes changes that have been made.
The description explains _WHAT_ and _WHY_ was made instead of _HOW_.
- [ ] The pull request title is treated as the final commit message.
The following pattern must be used: `IGNITE-XXXX Change summary` where `XXXX` - number of JIRA issue.
- [ ] A reviewer has been mentioned through the JIRA comments
(see [the Maintainers list](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers))
- [ ] The pull request has been checked by the Teamcity Bot and
the `green visa` attached to the JIRA ticket (see [TC.Bot: Check PR](https://mtcga.gridgain.com/prs.html))
### Notes
- [How to Contribute](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute)
- [Coding abbreviation rules](https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules)
- [Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines)
- [Apache Ignite Teamcity Bot](https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot)
If you need any help, please email dev@ignite.apache.org or ask anу advice on http://asf.slack.com _#ignite_ channel.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-25449 Add memory leak detection in LongRunningQueryTest for He… [ignite]
Posted by "asf-gitbox-commits (via GitHub)" <gi...@apache.org>.
asf-gitbox-commits closed pull request #12087: IGNITE-25449 Add memory leak detection in LongRunningQueryTest for He…
URL: https://github.com/apache/ignite/pull/12087
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-25449 Add memory leak detection in LongRunningQueryTest for He… [ignite]
Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #12087:
URL: https://github.com/apache/ignite/pull/12087#discussion_r2153561959
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -392,6 +415,102 @@ public void testBigResultDistributed() throws Exception {
checkBigResultSet();
}
+ /**
+ * Verifies that while a query is not fully fetched, its {@link H2QueryInfo} is kept in {@link HeavyQueriesTracker}
+ * on all cluster nodes and its {@link H2QueryInfo#isSuspended()} returns {@code true}. Then, once the query is fully
+ * fetched, its {@link H2QueryInfo} is removed from {@link HeavyQueriesTracker}.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithFullyFetchedIterator() {
+ Iterator<?> it = queryCursor(false).iterator();
+
+ checkQryInfoCount(gridCount());
+
+ H2QueryInfo qry = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qry.isSuspended());
+
+ it.forEachRemaining(x -> {});
+ }
+
+ /**
+ * Verifies that when the cursor of a not fully fetched query is closed, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithClosedCursor() {
+ FieldsQueryCursor<List<?>> cursor = queryCursor(false);
+
+ cursor.iterator().next();
+
+ checkQryInfoCount(gridCount());
+
+ H2QueryInfo qryInfo = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qryInfo.isSuspended());
+
+ cursor.close();
+ }
+
+ /**
+ * Verifies that when a not fully fetched query is cancelled, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithCancelledQuery() {
+ cancelQuery(runNotFullyFetchedQuery(false));
+ }
+
+ /**
+ * Verifies that when a local not fully fetched query is cancelled, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithCancelledLocalQuery() {
+ long qryId = runNotFullyFetchedQuery(true);
+
+ ((IgniteEx)ignite).context().query().cancelLocalQueries(Set.of(qryId));
+ }
+
+ /**
+ * Verifies that when there are multiple not fully fetched queries, and they are cancelled separately, corresponding
+ * {@link H2QueryInfo} instances are removed from {@link HeavyQueriesTracker} on all cluster nodes.
+ * */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithMultipleCancelledQueries() {
+ int qryCnt = 4;
+ int cnldQryCnt = 2;
+
+ for (int i = 0; i < qryCnt; i++)
+ runNotFullyFetchedQuery(false);
+
+ for (int i = 0; i < gridCount(); ++i)
+ assertEquals(qryCnt, heavyQueriesTracker(i).getQueries().size());
+
+ for (int i = 0; i < cnldQryCnt; i++)
+ cancelQuery(i + 1);
+
+ for (int i = 0; i < gridCount(); ++i) {
+ Set<TrackableQuery> qrys = heavyQueriesTracker(i).getQueries();
+
+ assertEquals(cnldQryCnt, qrys.size());
+
+ assertFalse(qrys.stream().anyMatch(qryInfo -> {
+ long id = ((H2QueryInfo)qryInfo).queryId();
+
+ return IntStream.range(0, cnldQryCnt).anyMatch(x -> x == id);
Review Comment:
Test refactored.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-25449 Add memory leak detection in LongRunningQueryTest for He… [ignite]
Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #12087:
URL: https://github.com/apache/ignite/pull/12087#discussion_r2153562534
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -583,24 +704,55 @@ public void runDml(String dml) {
assertTrue(lsnrDml.check());
}
+ /**
+ * @param loc Flag indicating if the query is local.
+ * @return Query id.
+ */
+ public long runNotFullyFetchedQuery(boolean loc) {
+ queryCursor(loc).iterator().next();
+
+ checkQryInfoCount(loc ? 1 : gridCount());
+
+ H2QueryInfo qryInfo = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qryInfo.isSuspended());
+
+ return qryInfo.queryId();
+ }
+
+ /**
+ * @param loc Flag indicating if the query is local.
+ * @return Query cursor.
+ */
+ public FieldsQueryCursor<List<?>> queryCursor(boolean loc) {
+ return ignite.cache("test").query(new SqlFieldsQuery("select * from test").setLocal(loc).setPageSize(1));
+ }
+
+ /**
+ * @param qryId Query id.
+ */
+ public void cancelQuery(long qryId) {
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-25449 Add memory leak detection in LongRunningQueryTest for He… [ignite]
Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #12087:
URL: https://github.com/apache/ignite/pull/12087#discussion_r2153559609
##########
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/twostep/GridMapQueryExecutor.java:
##########
@@ -176,6 +177,8 @@ public void onCancel(ClusterNode node, GridQueryCancelRequest msg) {
}
nodeRess.cancelRequest(qryReqId);
+
+ h2.heavyQueriesTracker().stopTracking(new H2QueryInfo(node.id(), qryReqId), null);
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-25449 Add memory leak detection in LongRunningQueryTest for He… [ignite]
Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #12087:
URL: https://github.com/apache/ignite/pull/12087#discussion_r2153579703
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/query/running/HeavyQueriesTracker.java:
##########
@@ -140,7 +140,7 @@ public void startTracking(TrackableQuery qryInfo) {
* @param qryInfo Query info to remove.
* @param err Exception if query executed with error.
*/
- public void stopTracking(TrackableQuery qryInfo, @Nullable Throwable err) {
+ public synchronized void stopTracking(TrackableQuery qryInfo, @Nullable Throwable err) {
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-25449 Add memory leak detection in LongRunningQueryTest for He… [ignite]
Posted by "alex-plekhanov (via GitHub)" <gi...@apache.org>.
alex-plekhanov commented on code in PR #12087:
URL: https://github.com/apache/ignite/pull/12087#discussion_r2152384040
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -583,24 +704,55 @@ public void runDml(String dml) {
assertTrue(lsnrDml.check());
}
+ /**
+ * @param loc Flag indicating if the query is local.
+ * @return Query id.
+ */
+ public long runNotFullyFetchedQuery(boolean loc) {
Review Comment:
private
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -583,24 +704,55 @@ public void runDml(String dml) {
assertTrue(lsnrDml.check());
}
+ /**
+ * @param loc Flag indicating if the query is local.
+ * @return Query id.
+ */
+ public long runNotFullyFetchedQuery(boolean loc) {
+ queryCursor(loc).iterator().next();
+
+ checkQryInfoCount(loc ? 1 : gridCount());
+
+ H2QueryInfo qryInfo = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qryInfo.isSuspended());
+
+ return qryInfo.queryId();
+ }
+
+ /**
+ * @param loc Flag indicating if the query is local.
+ * @return Query cursor.
+ */
+ public FieldsQueryCursor<List<?>> queryCursor(boolean loc) {
Review Comment:
private
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -583,24 +704,55 @@ public void runDml(String dml) {
assertTrue(lsnrDml.check());
}
+ /**
+ * @param loc Flag indicating if the query is local.
+ * @return Query id.
+ */
+ public long runNotFullyFetchedQuery(boolean loc) {
+ queryCursor(loc).iterator().next();
+
+ checkQryInfoCount(loc ? 1 : gridCount());
+
+ H2QueryInfo qryInfo = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qryInfo.isSuspended());
+
+ return qryInfo.queryId();
+ }
+
+ /**
+ * @param loc Flag indicating if the query is local.
+ * @return Query cursor.
+ */
+ public FieldsQueryCursor<List<?>> queryCursor(boolean loc) {
+ return ignite.cache("test").query(new SqlFieldsQuery("select * from test").setLocal(loc).setPageSize(1));
+ }
+
+ /**
+ * @param qryId Query id.
+ */
+ public void cancelQuery(long qryId) {
Review Comment:
private
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -643,7 +795,29 @@ private ListeningTestLogger testLog() {
* @return Heavy queries tracker.
*/
private HeavyQueriesTracker heavyQueriesTracker() {
- return ((IgniteH2Indexing)grid().context().query().getIndexing()).heavyQueriesTracker();
+ return heavyQueriesTracker(0);
+ }
+
+ /** */
+ private HeavyQueriesTracker heavyQueriesTracker(int idx) {
+ return ((IgniteH2Indexing)grid(idx).context().query().getIndexing()).heavyQueriesTracker();
+ }
+
+ /** */
+ private void checkQryInfoCount(int exp) {
+ int res = 0;
+
+ for (int i = 0; i < gridCount(); i++) {
+ if (!heavyQueriesTracker(i).getQueries().isEmpty())
+ res++;
+ }
+
+ assertEquals(exp, res);
+ }
+
+ /** */
+ public int gridCount() {
Review Comment:
private
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-25449 Add memory leak detection in LongRunningQueryTest for He… [ignite]
Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #12087:
URL: https://github.com/apache/ignite/pull/12087#discussion_r2153561772
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -643,7 +795,29 @@ private ListeningTestLogger testLog() {
* @return Heavy queries tracker.
*/
private HeavyQueriesTracker heavyQueriesTracker() {
- return ((IgniteH2Indexing)grid().context().query().getIndexing()).heavyQueriesTracker();
+ return heavyQueriesTracker(0);
+ }
+
+ /** */
+ private HeavyQueriesTracker heavyQueriesTracker(int idx) {
+ return ((IgniteH2Indexing)grid(idx).context().query().getIndexing()).heavyQueriesTracker();
+ }
+
+ /** */
+ private void checkQryInfoCount(int exp) {
+ int res = 0;
+
+ for (int i = 0; i < gridCount(); i++) {
+ if (!heavyQueriesTracker(i).getQueries().isEmpty())
+ res++;
+ }
Review Comment:
Test refactored.
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -392,6 +415,102 @@ public void testBigResultDistributed() throws Exception {
checkBigResultSet();
}
+ /**
+ * Verifies that while a query is not fully fetched, its {@link H2QueryInfo} is kept in {@link HeavyQueriesTracker}
+ * on all cluster nodes and its {@link H2QueryInfo#isSuspended()} returns {@code true}. Then, once the query is fully
+ * fetched, its {@link H2QueryInfo} is removed from {@link HeavyQueriesTracker}.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithFullyFetchedIterator() {
+ Iterator<?> it = queryCursor(false).iterator();
+
+ checkQryInfoCount(gridCount());
+
+ H2QueryInfo qry = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qry.isSuspended());
+
+ it.forEachRemaining(x -> {});
+ }
+
+ /**
+ * Verifies that when the cursor of a not fully fetched query is closed, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithClosedCursor() {
+ FieldsQueryCursor<List<?>> cursor = queryCursor(false);
+
+ cursor.iterator().next();
+
+ checkQryInfoCount(gridCount());
+
+ H2QueryInfo qryInfo = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qryInfo.isSuspended());
+
+ cursor.close();
+ }
+
+ /**
+ * Verifies that when a not fully fetched query is cancelled, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithCancelledQuery() {
+ cancelQuery(runNotFullyFetchedQuery(false));
+ }
+
+ /**
+ * Verifies that when a local not fully fetched query is cancelled, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithCancelledLocalQuery() {
+ long qryId = runNotFullyFetchedQuery(true);
+
+ ((IgniteEx)ignite).context().query().cancelLocalQueries(Set.of(qryId));
+ }
+
+ /**
+ * Verifies that when there are multiple not fully fetched queries, and they are cancelled separately, corresponding
+ * {@link H2QueryInfo} instances are removed from {@link HeavyQueriesTracker} on all cluster nodes.
+ * */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithMultipleCancelledQueries() {
+ int qryCnt = 4;
+ int cnldQryCnt = 2;
+
+ for (int i = 0; i < qryCnt; i++)
+ runNotFullyFetchedQuery(false);
+
+ for (int i = 0; i < gridCount(); ++i)
+ assertEquals(qryCnt, heavyQueriesTracker(i).getQueries().size());
+
+ for (int i = 0; i < cnldQryCnt; i++)
+ cancelQuery(i + 1);
Review Comment:
Done.
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -392,6 +415,102 @@ public void testBigResultDistributed() throws Exception {
checkBigResultSet();
}
+ /**
+ * Verifies that while a query is not fully fetched, its {@link H2QueryInfo} is kept in {@link HeavyQueriesTracker}
+ * on all cluster nodes and its {@link H2QueryInfo#isSuspended()} returns {@code true}. Then, once the query is fully
+ * fetched, its {@link H2QueryInfo} is removed from {@link HeavyQueriesTracker}.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithFullyFetchedIterator() {
+ Iterator<?> it = queryCursor(false).iterator();
+
+ checkQryInfoCount(gridCount());
+
+ H2QueryInfo qry = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qry.isSuspended());
+
+ it.forEachRemaining(x -> {});
+ }
+
+ /**
+ * Verifies that when the cursor of a not fully fetched query is closed, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithClosedCursor() {
+ FieldsQueryCursor<List<?>> cursor = queryCursor(false);
+
+ cursor.iterator().next();
+
+ checkQryInfoCount(gridCount());
+
+ H2QueryInfo qryInfo = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qryInfo.isSuspended());
+
+ cursor.close();
+ }
+
+ /**
+ * Verifies that when a not fully fetched query is cancelled, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithCancelledQuery() {
+ cancelQuery(runNotFullyFetchedQuery(false));
+ }
+
+ /**
+ * Verifies that when a local not fully fetched query is cancelled, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithCancelledLocalQuery() {
+ long qryId = runNotFullyFetchedQuery(true);
+
+ ((IgniteEx)ignite).context().query().cancelLocalQueries(Set.of(qryId));
+ }
+
+ /**
+ * Verifies that when there are multiple not fully fetched queries, and they are cancelled separately, corresponding
+ * {@link H2QueryInfo} instances are removed from {@link HeavyQueriesTracker} on all cluster nodes.
+ * */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithMultipleCancelledQueries() {
+ int qryCnt = 4;
+ int cnldQryCnt = 2;
+
+ for (int i = 0; i < qryCnt; i++)
+ runNotFullyFetchedQuery(false);
+
+ for (int i = 0; i < gridCount(); ++i)
+ assertEquals(qryCnt, heavyQueriesTracker(i).getQueries().size());
+
+ for (int i = 0; i < cnldQryCnt; i++)
+ cancelQuery(i + 1);
+
+ for (int i = 0; i < gridCount(); ++i) {
+ Set<TrackableQuery> qrys = heavyQueriesTracker(i).getQueries();
+
+ assertEquals(cnldQryCnt, qrys.size());
Review Comment:
Check corrected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-25449 Add memory leak detection in LongRunningQueryTest for He… [ignite]
Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #12087:
URL: https://github.com/apache/ignite/pull/12087#discussion_r2153562315
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -583,24 +704,55 @@ public void runDml(String dml) {
assertTrue(lsnrDml.check());
}
+ /**
+ * @param loc Flag indicating if the query is local.
+ * @return Query id.
+ */
+ public long runNotFullyFetchedQuery(boolean loc) {
Review Comment:
Done.
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -583,24 +704,55 @@ public void runDml(String dml) {
assertTrue(lsnrDml.check());
}
+ /**
+ * @param loc Flag indicating if the query is local.
+ * @return Query id.
+ */
+ public long runNotFullyFetchedQuery(boolean loc) {
+ queryCursor(loc).iterator().next();
+
+ checkQryInfoCount(loc ? 1 : gridCount());
+
+ H2QueryInfo qryInfo = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qryInfo.isSuspended());
+
+ return qryInfo.queryId();
+ }
+
+ /**
+ * @param loc Flag indicating if the query is local.
+ * @return Query cursor.
+ */
+ public FieldsQueryCursor<List<?>> queryCursor(boolean loc) {
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-25449 Add memory leak detection in LongRunningQueryTest for He… [ignite]
Posted by "oleg-vlsk (via GitHub)" <gi...@apache.org>.
oleg-vlsk commented on code in PR #12087:
URL: https://github.com/apache/ignite/pull/12087#discussion_r2153562861
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -643,7 +795,29 @@ private ListeningTestLogger testLog() {
* @return Heavy queries tracker.
*/
private HeavyQueriesTracker heavyQueriesTracker() {
- return ((IgniteH2Indexing)grid().context().query().getIndexing()).heavyQueriesTracker();
+ return heavyQueriesTracker(0);
+ }
+
+ /** */
+ private HeavyQueriesTracker heavyQueriesTracker(int idx) {
+ return ((IgniteH2Indexing)grid(idx).context().query().getIndexing()).heavyQueriesTracker();
+ }
+
+ /** */
+ private void checkQryInfoCount(int exp) {
+ int res = 0;
+
+ for (int i = 0; i < gridCount(); i++) {
+ if (!heavyQueriesTracker(i).getQueries().isEmpty())
+ res++;
+ }
+
+ assertEquals(exp, res);
+ }
+
+ /** */
+ public int gridCount() {
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-25449 Add memory leak detection in LongRunningQueryTest for He… [ignite]
Posted by "alex-plekhanov (via GitHub)" <gi...@apache.org>.
alex-plekhanov commented on code in PR #12087:
URL: https://github.com/apache/ignite/pull/12087#discussion_r2151760534
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/query/running/HeavyQueriesTracker.java:
##########
@@ -140,7 +140,7 @@ public void startTracking(TrackableQuery qryInfo) {
* @param qryInfo Query info to remove.
* @param err Exception if query executed with error.
*/
- public void stopTracking(TrackableQuery qryInfo, @Nullable Throwable err) {
+ public synchronized void stopTracking(TrackableQuery qryInfo, @Nullable Throwable err) {
Review Comment:
qrys is thread safe collection, no synchronization required here. But to avoid double logging you can add somethink like `if (qrys.remove(qryInfo) != null) { ... log ... }`. Synchronization doesn't solve the double logging porblem.
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -392,6 +415,102 @@ public void testBigResultDistributed() throws Exception {
checkBigResultSet();
}
+ /**
+ * Verifies that while a query is not fully fetched, its {@link H2QueryInfo} is kept in {@link HeavyQueriesTracker}
+ * on all cluster nodes and its {@link H2QueryInfo#isSuspended()} returns {@code true}. Then, once the query is fully
+ * fetched, its {@link H2QueryInfo} is removed from {@link HeavyQueriesTracker}.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithFullyFetchedIterator() {
+ Iterator<?> it = queryCursor(false).iterator();
+
+ checkQryInfoCount(gridCount());
+
+ H2QueryInfo qry = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qry.isSuspended());
+
+ it.forEachRemaining(x -> {});
+ }
+
+ /**
+ * Verifies that when the cursor of a not fully fetched query is closed, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithClosedCursor() {
+ FieldsQueryCursor<List<?>> cursor = queryCursor(false);
+
+ cursor.iterator().next();
+
+ checkQryInfoCount(gridCount());
+
+ H2QueryInfo qryInfo = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qryInfo.isSuspended());
+
+ cursor.close();
+ }
+
+ /**
+ * Verifies that when a not fully fetched query is cancelled, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithCancelledQuery() {
+ cancelQuery(runNotFullyFetchedQuery(false));
+ }
+
+ /**
+ * Verifies that when a local not fully fetched query is cancelled, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithCancelledLocalQuery() {
+ long qryId = runNotFullyFetchedQuery(true);
+
+ ((IgniteEx)ignite).context().query().cancelLocalQueries(Set.of(qryId));
+ }
+
+ /**
+ * Verifies that when there are multiple not fully fetched queries, and they are cancelled separately, corresponding
+ * {@link H2QueryInfo} instances are removed from {@link HeavyQueriesTracker} on all cluster nodes.
+ * */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithMultipleCancelledQueries() {
+ int qryCnt = 4;
+ int cnldQryCnt = 2;
+
+ for (int i = 0; i < qryCnt; i++)
+ runNotFullyFetchedQuery(false);
+
+ for (int i = 0; i < gridCount(); ++i)
+ assertEquals(qryCnt, heavyQueriesTracker(i).getQueries().size());
+
+ for (int i = 0; i < cnldQryCnt; i++)
+ cancelQuery(i + 1);
+
+ for (int i = 0; i < gridCount(); ++i) {
+ Set<TrackableQuery> qrys = heavyQueriesTracker(i).getQueries();
+
+ assertEquals(cnldQryCnt, qrys.size());
Review Comment:
qryCnt - cnldQryCnt
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -392,6 +415,102 @@ public void testBigResultDistributed() throws Exception {
checkBigResultSet();
}
+ /**
+ * Verifies that while a query is not fully fetched, its {@link H2QueryInfo} is kept in {@link HeavyQueriesTracker}
+ * on all cluster nodes and its {@link H2QueryInfo#isSuspended()} returns {@code true}. Then, once the query is fully
+ * fetched, its {@link H2QueryInfo} is removed from {@link HeavyQueriesTracker}.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithFullyFetchedIterator() {
+ Iterator<?> it = queryCursor(false).iterator();
+
+ checkQryInfoCount(gridCount());
+
+ H2QueryInfo qry = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qry.isSuspended());
+
+ it.forEachRemaining(x -> {});
+ }
+
+ /**
+ * Verifies that when the cursor of a not fully fetched query is closed, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithClosedCursor() {
+ FieldsQueryCursor<List<?>> cursor = queryCursor(false);
+
+ cursor.iterator().next();
+
+ checkQryInfoCount(gridCount());
+
+ H2QueryInfo qryInfo = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qryInfo.isSuspended());
+
+ cursor.close();
+ }
+
+ /**
+ * Verifies that when a not fully fetched query is cancelled, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithCancelledQuery() {
+ cancelQuery(runNotFullyFetchedQuery(false));
+ }
+
+ /**
+ * Verifies that when a local not fully fetched query is cancelled, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithCancelledLocalQuery() {
+ long qryId = runNotFullyFetchedQuery(true);
+
+ ((IgniteEx)ignite).context().query().cancelLocalQueries(Set.of(qryId));
+ }
+
+ /**
+ * Verifies that when there are multiple not fully fetched queries, and they are cancelled separately, corresponding
+ * {@link H2QueryInfo} instances are removed from {@link HeavyQueriesTracker} on all cluster nodes.
+ * */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithMultipleCancelledQueries() {
+ int qryCnt = 4;
+ int cnldQryCnt = 2;
+
+ for (int i = 0; i < qryCnt; i++)
+ runNotFullyFetchedQuery(false);
+
+ for (int i = 0; i < gridCount(); ++i)
+ assertEquals(qryCnt, heavyQueriesTracker(i).getQueries().size());
+
+ for (int i = 0; i < cnldQryCnt; i++)
+ cancelQuery(i + 1);
Review Comment:
It's not safe to rely on internals of id generation, let's use ids from heavyQueriesTracker(i).getQueries()
##########
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/twostep/GridMapQueryExecutor.java:
##########
@@ -176,6 +177,8 @@ public void onCancel(ClusterNode node, GridQueryCancelRequest msg) {
}
nodeRess.cancelRequest(qryReqId);
+
+ h2.heavyQueriesTracker().stopTracking(new H2QueryInfo(node.id(), qryReqId), null);
Review Comment:
Consider to call stopTracking from MapQueryResult.Result#close. There we have qryInfo and looks like it's safer, in case someone add any new cancel request method.
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -392,6 +415,102 @@ public void testBigResultDistributed() throws Exception {
checkBigResultSet();
}
+ /**
+ * Verifies that while a query is not fully fetched, its {@link H2QueryInfo} is kept in {@link HeavyQueriesTracker}
+ * on all cluster nodes and its {@link H2QueryInfo#isSuspended()} returns {@code true}. Then, once the query is fully
+ * fetched, its {@link H2QueryInfo} is removed from {@link HeavyQueriesTracker}.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithFullyFetchedIterator() {
+ Iterator<?> it = queryCursor(false).iterator();
+
+ checkQryInfoCount(gridCount());
+
+ H2QueryInfo qry = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qry.isSuspended());
+
+ it.forEachRemaining(x -> {});
+ }
+
+ /**
+ * Verifies that when the cursor of a not fully fetched query is closed, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithClosedCursor() {
+ FieldsQueryCursor<List<?>> cursor = queryCursor(false);
+
+ cursor.iterator().next();
+
+ checkQryInfoCount(gridCount());
+
+ H2QueryInfo qryInfo = (H2QueryInfo)heavyQueriesTracker().getQueries().iterator().next();
+
+ assertTrue(qryInfo.isSuspended());
+
+ cursor.close();
+ }
+
+ /**
+ * Verifies that when a not fully fetched query is cancelled, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithCancelledQuery() {
+ cancelQuery(runNotFullyFetchedQuery(false));
+ }
+
+ /**
+ * Verifies that when a local not fully fetched query is cancelled, its {@link H2QueryInfo} is removed from
+ * {@link HeavyQueriesTracker} on all cluster nodes.
+ */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithCancelledLocalQuery() {
+ long qryId = runNotFullyFetchedQuery(true);
+
+ ((IgniteEx)ignite).context().query().cancelLocalQueries(Set.of(qryId));
+ }
+
+ /**
+ * Verifies that when there are multiple not fully fetched queries, and they are cancelled separately, corresponding
+ * {@link H2QueryInfo} instances are removed from {@link HeavyQueriesTracker} on all cluster nodes.
+ * */
+ @Test
+ @MultiNodeTest
+ public void testEmptyHeavyQueriesTrackerWithMultipleCancelledQueries() {
+ int qryCnt = 4;
+ int cnldQryCnt = 2;
+
+ for (int i = 0; i < qryCnt; i++)
+ runNotFullyFetchedQuery(false);
+
+ for (int i = 0; i < gridCount(); ++i)
+ assertEquals(qryCnt, heavyQueriesTracker(i).getQueries().size());
+
+ for (int i = 0; i < cnldQryCnt; i++)
+ cancelQuery(i + 1);
+
+ for (int i = 0; i < gridCount(); ++i) {
+ Set<TrackableQuery> qrys = heavyQueriesTracker(i).getQueries();
+
+ assertEquals(cnldQryCnt, qrys.size());
+
+ assertFalse(qrys.stream().anyMatch(qryInfo -> {
+ long id = ((H2QueryInfo)qryInfo).queryId();
+
+ return IntStream.range(0, cnldQryCnt).anyMatch(x -> x == id);
Review Comment:
Wrong check, you cancel id: `i+1`, but check `i`
##########
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/LongRunningQueryTest.java:
##########
@@ -643,7 +795,29 @@ private ListeningTestLogger testLog() {
* @return Heavy queries tracker.
*/
private HeavyQueriesTracker heavyQueriesTracker() {
- return ((IgniteH2Indexing)grid().context().query().getIndexing()).heavyQueriesTracker();
+ return heavyQueriesTracker(0);
+ }
+
+ /** */
+ private HeavyQueriesTracker heavyQueriesTracker(int idx) {
+ return ((IgniteH2Indexing)grid(idx).context().query().getIndexing()).heavyQueriesTracker();
+ }
+
+ /** */
+ private void checkQryInfoCount(int exp) {
+ int res = 0;
+
+ for (int i = 0; i < gridCount(); i++) {
+ if (!heavyQueriesTracker(i).getQueries().isEmpty())
+ res++;
+ }
Review Comment:
res += heavyQueriesTracker(i).getQueries().size();
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org