You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "alex-plekhanov (via GitHub)" <gi...@apache.org> on 2026/04/29 10:31:10 UTC
[PR] IGNITE-28623 SQL Calcite: Fix premature orphan iterator closing by GC [ignite]
alex-plekhanov opened a new pull request, #13089:
URL: https://github.com/apache/ignite/pull/13089
Thank you for submitting the pull request to the Apache Ignite.
In order to streamline the review of the contribution
we ask you to ensure the following steps have been taken:
### The Contribution Checklist
- [ ] There is a single JIRA ticket related to the pull request.
- [ ] The web-link to the pull request is attached to the JIRA ticket.
- [ ] The JIRA ticket has the _Patch Available_ state.
- [ ] The pull request body describes changes that have been made.
The description explains _WHAT_ and _WHY_ was made instead of _HOW_.
- [ ] The pull request title is treated as the final commit message.
The following pattern must be used: `IGNITE-XXXX Change summary` where `XXXX` - number of JIRA issue.
- [ ] A reviewer has been mentioned through the JIRA comments
(see [the Maintainers list](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers))
- [ ] The pull request has been checked by the Teamcity Bot and
the `green visa` attached to the JIRA ticket (see tab `PR Check` at [TC.Bot - Instance 1](https://tcbot2.sbt-ignite-dev.ru/prs.html) or [TC.Bot - Instance 2](https://mtcga.gridgain.com/prs.html))
### Notes
- [How to Contribute](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute)
- [Coding abbreviation rules](https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules)
- [Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines)
- [Apache Ignite Teamcity Bot](https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot)
If you need any help, please email dev@ignite.apache.org or ask anу advice on http://asf.slack.com _#ignite_ channel.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-28623 SQL Calcite: Fix premature orphan iterator closing by GC [ignite]
Posted by "alex-plekhanov (via GitHub)" <gi...@apache.org>.
alex-plekhanov commented on code in PR #13089:
URL: https://github.com/apache/ignite/pull/13089#discussion_r3187201041
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/ClosableIteratorsHolder.java:
##########
@@ -97,18 +97,30 @@ private Reference nextRef(boolean blocking) {
}
/** */
- private AutoCloseable closeable(Object referent, Object resource) {
- if (!(resource instanceof AutoCloseable))
+ private AutoCloseable closeable(Object referent, Object rsrc) {
+ if (!(rsrc instanceof AutoCloseable))
return null;
- return new CloseableReference(referent, resource);
+ CloseableReference ref = new CloseableReference(referent);
+
+ refMap.put(ref, rsrc);
+
+ return ref;
}
/** */
private final class DelegatingIterator<T> implements Iterator<T>, AutoCloseable {
/** */
private final Iterator<T> delegate;
+ /**
+ * This variable is required to keep reference to current instance while delegate call
+ * (hasNext/next/remove/forEachRemaining) is not completed. We actually don't care about variable value and
+ * thread safety, it's only to prevent premature garbage collection and iterator closing before returning
+ * result to the user.
+ */
+ private boolean inUse;
Review Comment:
It's not debug related flag. There is a javadoc comment describing why this flag is needed. It not masks the exception it prevents to garbage-collect iterator object before delegate object returns result.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-28623 SQL Calcite: Fix premature orphan iterator closing by GC [ignite]
Posted by "zstan (via GitHub)" <gi...@apache.org>.
zstan commented on code in PR #13089:
URL: https://github.com/apache/ignite/pull/13089#discussion_r3186687544
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/RootNode.java:
##########
@@ -324,9 +321,7 @@ private void checkException() {
if (e == null)
return;
- if (e instanceof IgniteSQLException)
- throw (IgniteSQLException)e;
- else
- throw new IgniteSQLException("An error occurred while query executing - " + e.getMessage(), IgniteQueryErrorCode.UNKNOWN, e);
+ throw new IgniteSQLException("An error occurred while query executing - " + e.getMessage(),
+ e instanceof IgniteSQLException ? ((IgniteSQLException)e).statusCode() : IgniteQueryErrorCode.UNKNOWN, e);
Review Comment:
probably not to re-pack but use approach like :
```
@Override public boolean hasNext() {
...
sneakyThrow(e);
}
public static <E extends Throwable> E sneakyThrow(Throwable e) throws E {
throw (E) e;
}
```
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/RootNode.java:
##########
@@ -205,8 +204,6 @@ public UUID queryId() {
/** {@inheritDoc} */
@Override public boolean hasNext() {
- checkException();
Review Comment:
why this check need to be removed ?
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/ClosableIteratorsHolder.java:
##########
@@ -97,18 +97,30 @@ private Reference nextRef(boolean blocking) {
}
/** */
- private AutoCloseable closeable(Object referent, Object resource) {
- if (!(resource instanceof AutoCloseable))
+ private AutoCloseable closeable(Object referent, Object rsrc) {
Review Comment:
```suggestion
@Nullable private AutoCloseable closeable(Object referent, Iterator<?> rsrc) {
```
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/ClosableIteratorsHolder.java:
##########
@@ -119,37 +131,66 @@ private DelegatingIterator(Iterator<T> delegate) {
/** {@inheritDoc} */
@Override public boolean hasNext() {
- return delegate.hasNext();
+ inUse = true;
+
+ try {
+ return delegate.hasNext();
+ }
+ finally {
+ inUse = false;
+ }
}
/** {@inheritDoc} */
@Override public T next() {
- return delegate.next();
+ inUse = true;
+
+ try {
+ return delegate.next();
+ }
+ finally {
+ inUse = false;
+ }
}
/** {@inheritDoc} */
@Override public void remove() {
- delegate.remove();
+ inUse = true;
+
+ try {
+ delegate.remove();
+ }
+ finally {
+ inUse = false;
+ }
}
/** {@inheritDoc} */
@Override public void forEachRemaining(Consumer<? super T> action) {
- delegate.forEachRemaining(action);
+ inUse = true;
+
+ try {
+ delegate.forEachRemaining(action);
+ }
+ finally {
+ inUse = false;
+ }
}
/** {@inheritDoc} */
@Override public void close() throws Exception {
+ if (log.isDebugEnabled())
+ log.debug("Closing iterator [delegate=" + delegate + ", inUse=" + inUse + ']');
+
Commons.close(closeable);
}
}
/** */
private final class CloseableReference extends WeakReference implements AutoCloseable {
/** */
- private CloseableReference(Object referent, Object resource) {
+ private CloseableReference(Object referent) {
Review Comment:
probably we need to remove WeakReference related code and create more deterministoc model ? there is also no weak ref in ignite3
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/ClosableIteratorsHolder.java:
##########
@@ -97,18 +97,30 @@ private Reference nextRef(boolean blocking) {
}
/** */
- private AutoCloseable closeable(Object referent, Object resource) {
- if (!(resource instanceof AutoCloseable))
+ private AutoCloseable closeable(Object referent, Object rsrc) {
+ if (!(rsrc instanceof AutoCloseable))
return null;
- return new CloseableReference(referent, resource);
+ CloseableReference ref = new CloseableReference(referent);
+
+ refMap.put(ref, rsrc);
+
+ return ref;
}
/** */
private final class DelegatingIterator<T> implements Iterator<T>, AutoCloseable {
/** */
private final Iterator<T> delegate;
+ /**
+ * This variable is required to keep reference to current instance while delegate call
+ * (hasNext/next/remove/forEachRemaining) is not completed. We actually don't care about variable value and
+ * thread safety, it's only to prevent premature garbage collection and iterator closing before returning
+ * result to the user.
+ */
+ private boolean inUse;
Review Comment:
debug related flag in production code - looks weird for me
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/ClosableIteratorsHolder.java:
##########
@@ -97,18 +97,30 @@ private Reference nextRef(boolean blocking) {
}
/** */
- private AutoCloseable closeable(Object referent, Object resource) {
- if (!(resource instanceof AutoCloseable))
+ private AutoCloseable closeable(Object referent, Object rsrc) {
+ if (!(rsrc instanceof AutoCloseable))
return null;
- return new CloseableReference(referent, resource);
+ CloseableReference ref = new CloseableReference(referent);
+
+ refMap.put(ref, rsrc);
+
+ return ref;
}
/** */
private final class DelegatingIterator<T> implements Iterator<T>, AutoCloseable {
/** */
private final Iterator<T> delegate;
+ /**
+ * This variable is required to keep reference to current instance while delegate call
+ * (hasNext/next/remove/forEachRemaining) is not completed. We actually don't care about variable value and
+ * thread safety, it's only to prevent premature garbage collection and iterator closing before returning
+ * result to the user.
+ */
+ private boolean inUse;
Review Comment:
also i wonder ) i just comment this: //private boolean inUse;
and get already known : Suppressed: [CIRCULAR REFERENCE:
seems this flag mask it somehow ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-28623 SQL Calcite: Fix premature orphan iterator closing by GC [ignite]
Posted by "alex-plekhanov (via GitHub)" <gi...@apache.org>.
alex-plekhanov commented on code in PR #13089:
URL: https://github.com/apache/ignite/pull/13089#discussion_r3187181134
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/RootNode.java:
##########
@@ -324,9 +321,7 @@ private void checkException() {
if (e == null)
return;
- if (e instanceof IgniteSQLException)
- throw (IgniteSQLException)e;
- else
- throw new IgniteSQLException("An error occurred while query executing - " + e.getMessage(), IgniteQueryErrorCode.UNKNOWN, e);
+ throw new IgniteSQLException("An error occurred while query executing - " + e.getMessage(),
+ e instanceof IgniteSQLException ? ((IgniteSQLException)e).statusCode() : IgniteQueryErrorCode.UNKNOWN, e);
Review Comment:
Here repacking is required to have correct stack trace in exception. WIthout repacking we just have cancellation stack trace.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-28623 SQL Calcite: Fix premature orphan iterator closing by GC [ignite]
Posted by "asf-gitbox-commits (via GitHub)" <gi...@apache.org>.
asf-gitbox-commits closed pull request #13089: IGNITE-28623 SQL Calcite: Fix premature orphan iterator closing by GC
URL: https://github.com/apache/ignite/pull/13089
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-28623 SQL Calcite: Fix premature orphan iterator closing by GC [ignite]
Posted by "alex-plekhanov (via GitHub)" <gi...@apache.org>.
alex-plekhanov commented on code in PR #13089:
URL: https://github.com/apache/ignite/pull/13089#discussion_r3187237703
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/ClosableIteratorsHolder.java:
##########
@@ -97,18 +97,30 @@ private Reference nextRef(boolean blocking) {
}
/** */
- private AutoCloseable closeable(Object referent, Object resource) {
- if (!(resource instanceof AutoCloseable))
+ private AutoCloseable closeable(Object referent, Object rsrc) {
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-28623 SQL Calcite: Fix premature orphan iterator closing by GC [ignite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #13089:
URL: https://github.com/apache/ignite/pull/13089#issuecomment-4377984871
## [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=13089) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=13089&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=13089&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=13089&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=13089&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=13089&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=13089)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-28623 SQL Calcite: Fix premature orphan iterator closing by GC [ignite]
Posted by "alex-plekhanov (via GitHub)" <gi...@apache.org>.
alex-plekhanov commented on code in PR #13089:
URL: https://github.com/apache/ignite/pull/13089#discussion_r3187189888
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/RootNode.java:
##########
@@ -205,8 +204,6 @@ public UUID queryId() {
/** {@inheritDoc} */
@Override public boolean hasNext() {
- checkException();
Review Comment:
We can return to user rows that was produced before query was cancelled. For example, if user want just one row.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-28623 SQL Calcite: Fix premature orphan iterator closing by GC [ignite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #13089:
URL: https://github.com/apache/ignite/pull/13089#issuecomment-4350100990
## [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=13089) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=13089&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=13089&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=13089&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=13089&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=13089&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=13089)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-28623 SQL Calcite: Fix premature orphan iterator closing by GC [ignite]
Posted by "alex-plekhanov (via GitHub)" <gi...@apache.org>.
alex-plekhanov commented on code in PR #13089:
URL: https://github.com/apache/ignite/pull/13089#discussion_r3187220490
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/ClosableIteratorsHolder.java:
##########
@@ -119,37 +131,66 @@ private DelegatingIterator(Iterator<T> delegate) {
/** {@inheritDoc} */
@Override public boolean hasNext() {
- return delegate.hasNext();
+ inUse = true;
+
+ try {
+ return delegate.hasNext();
+ }
+ finally {
+ inUse = false;
+ }
}
/** {@inheritDoc} */
@Override public T next() {
- return delegate.next();
+ inUse = true;
+
+ try {
+ return delegate.next();
+ }
+ finally {
+ inUse = false;
+ }
}
/** {@inheritDoc} */
@Override public void remove() {
- delegate.remove();
+ inUse = true;
+
+ try {
+ delegate.remove();
+ }
+ finally {
+ inUse = false;
+ }
}
/** {@inheritDoc} */
@Override public void forEachRemaining(Consumer<? super T> action) {
- delegate.forEachRemaining(action);
+ inUse = true;
+
+ try {
+ delegate.forEachRemaining(action);
+ }
+ finally {
+ inUse = false;
+ }
}
/** {@inheritDoc} */
@Override public void close() throws Exception {
+ if (log.isDebugEnabled())
+ log.debug("Closing iterator [delegate=" + delegate + ", inUse=" + inUse + ']');
+
Commons.close(closeable);
}
}
/** */
private final class CloseableReference extends WeakReference implements AutoCloseable {
/** */
- private CloseableReference(Object referent, Object resource) {
+ private CloseableReference(Object referent) {
Review Comment:
WakReference related code is a protection from query leaking. Without this code there can be unclosed queries in memory if user doesn't close cursor properly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-28623 SQL Calcite: Fix premature orphan iterator closing by GC [ignite]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #13089:
URL: https://github.com/apache/ignite/pull/13089#issuecomment-4342934940
## [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=13089) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=13089&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=13089&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=13089&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=13089&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=13089&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=13089)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org