You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by GitBox <gi...@apache.org> on 2022/04/22 19:33:04 UTC

[GitHub] [solr] magibney opened a new pull request, #818: SOLR-16163: Review unintended try-with-resources closing-order modifications

magibney opened a new pull request, #818:
URL: https://github.com/apache/solr/pull/818

   See: [SOLR-16163](https://issues.apache.org/jira/browse/SOLR-16163)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] risdenk commented on a diff in pull request #818: SOLR-16163: Review unintended try-with-resources closing-order modifications

Posted by GitBox <gi...@apache.org>.
risdenk commented on code in PR #818:
URL: https://github.com/apache/solr/pull/818#discussion_r866261735


##########
solr/core/src/java/org/apache/solr/client/solrj/embedded/EmbeddedSolrServer.java:
##########
@@ -274,6 +273,11 @@ public void writeResults(ResultContext ctx, JavaBinCodec codec) throws IOExcepti
         req.close();
         SolrRequestInfo.clearRequestInfo();
       }
+      // NOTE: `req` must be closed _before_ core, because `req` can hold a reference to
+      // SolrIndexSearcher which can in some cases block `core.close()` from completing
+      // successfully (by holding a reference to a core Directory). Hence we cannot trivially
+      // use try-with-resources to close `core`.
+      core.close();

Review Comment:
   same comment as below: couldn't `req = _parser.buildRequestFrom(core, params, getContentStreams(request));` be a try w/ resources instead of trying to do that outside with solrcore?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] risdenk commented on a diff in pull request #818: SOLR-16163: Review unintended try-with-resources closing-order modifications

Posted by GitBox <gi...@apache.org>.
risdenk commented on code in PR #818:
URL: https://github.com/apache/solr/pull/818#discussion_r866261735


##########
solr/core/src/java/org/apache/solr/client/solrj/embedded/EmbeddedSolrServer.java:
##########
@@ -274,6 +273,11 @@ public void writeResults(ResultContext ctx, JavaBinCodec codec) throws IOExcepti
         req.close();
         SolrRequestInfo.clearRequestInfo();
       }
+      // NOTE: `req` must be closed _before_ core, because `req` can hold a reference to
+      // SolrIndexSearcher which can in some cases block `core.close()` from completing
+      // successfully (by holding a reference to a core Directory). Hence we cannot trivially
+      // use try-with-resources to close `core`.
+      core.close();

Review Comment:
   couldn't `req = _parser.buildRequestFrom(core, params, getContentStreams(request));` be a try w/ resources instead of trying to do that outside with solrcore?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] dsmiley commented on a diff in pull request #818: SOLR-16163: Review unintended try-with-resources closing-order modifications

Posted by GitBox <gi...@apache.org>.
dsmiley commented on code in PR #818:
URL: https://github.com/apache/solr/pull/818#discussion_r866276174


##########
solr/core/src/java/org/apache/solr/client/solrj/embedded/EmbeddedSolrServer.java:
##########
@@ -274,6 +273,11 @@ public void writeResults(ResultContext ctx, JavaBinCodec codec) throws IOExcepti
         req.close();
         SolrRequestInfo.clearRequestInfo();
       }
+      // NOTE: `req` must be closed _before_ core, because `req` can hold a reference to
+      // SolrIndexSearcher which can in some cases block `core.close()` from completing
+      // successfully (by holding a reference to a core Directory). Hence we cannot trivially
+      // use try-with-resources to close `core`.
+      core.close();

Review Comment:
   Yes I agree; I think we should embrace try-with-resources as it provides structure.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] dsmiley commented on a diff in pull request #818: SOLR-16163: Review unintended try-with-resources closing-order modifications

Posted by GitBox <gi...@apache.org>.
dsmiley commented on code in PR #818:
URL: https://github.com/apache/solr/pull/818#discussion_r866276621


##########
solr/core/src/java/org/apache/solr/handler/admin/RequestApplyUpdatesOp.java:
##########
@@ -67,6 +69,11 @@ public void execute(CoreAdminHandler.CallInfo it) throws Exception {
             SolrException.ErrorCode.SERVER_ERROR, "Could not apply buffered updates", e);
     } finally {
       if (it.req != null) it.req.close();
+      // NOTE: `req` must be closed _before_ core, because `req` can hold a reference to
+      // SolrIndexSearcher which can in some cases block `core.close()` from completing
+      // successfully (by holding a reference to a core Directory). Hence we cannot trivially
+      // use try-with-resources to close `core`.

Review Comment:
   +1 Kevin; we agree!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] SOLR-16163: Review unintended try-with-resources closing-order modifications [solr]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #818:
URL: https://github.com/apache/solr/pull/818#issuecomment-1953292511

   This PR had no visible activity in the past 60 days, labeling it as stale. Any new activity will remove the stale label. To attract more reviewers, please tag someone or notify the dev@solr.apache.org mailing list. Thank you for your contribution!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] risdenk commented on a diff in pull request #818: SOLR-16163: Review unintended try-with-resources closing-order modifications

Posted by GitBox <gi...@apache.org>.
risdenk commented on code in PR #818:
URL: https://github.com/apache/solr/pull/818#discussion_r866259808


##########
solr/core/src/java/org/apache/solr/handler/admin/RequestApplyUpdatesOp.java:
##########
@@ -67,6 +69,11 @@ public void execute(CoreAdminHandler.CallInfo it) throws Exception {
             SolrException.ErrorCode.SERVER_ERROR, "Could not apply buffered updates", e);
     } finally {
       if (it.req != null) it.req.close();
+      // NOTE: `req` must be closed _before_ core, because `req` can hold a reference to
+      // SolrIndexSearcher which can in some cases block `core.close()` from completing
+      // successfully (by holding a reference to a core Directory). Hence we cannot trivially
+      // use try-with-resources to close `core`.

Review Comment:
   I think I'd use nested `try w/ resources` for SolrCore and inner `try/finally` for `it.req` that would force the order as well and make it more clear?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] risdenk commented on a diff in pull request #818: SOLR-16163: Review unintended try-with-resources closing-order modifications

Posted by GitBox <gi...@apache.org>.
risdenk commented on code in PR #818:
URL: https://github.com/apache/solr/pull/818#discussion_r866258661


##########
solr/core/src/java/org/apache/solr/handler/admin/RequestSyncShardOp.java:
##########
@@ -93,6 +93,7 @@ public void execute(CallInfo it) throws Exception {
       }
     } finally {
       // no recoveryStrat close for now
+      // NOTE: try-with-resources is fine here; ok for `core` to be closed before `syncStrategy`
       if (syncStrategy != null) {
         syncStrategy.close();

Review Comment:
   shouldn't `syncStrategy` be a try w/ resources inside `core != null` around line 58?
   
   it seems weird at least looking at the change on github that syncStrategy is even closed outside of where its used?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] risdenk commented on pull request #818: SOLR-16163: Review unintended try-with-resources closing-order modifications

Posted by GitBox <gi...@apache.org>.
risdenk commented on PR #818:
URL: https://github.com/apache/solr/pull/818#issuecomment-1118987748

   So my overall comment at looking at the underlying cause of the "out of order" is that it would be more clear to move the closes that order matters closer to where they should be closed (nested try w/ resources or otherwise) instead of trying to do multiple close in a finally.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


[GitHub] [solr] magibney commented on pull request #818: SOLR-16163: Review unintended try-with-resources closing-order modifications

Posted by GitBox <gi...@apache.org>.
magibney commented on PR #818:
URL: https://github.com/apache/solr/pull/818#issuecomment-1112654117

   From @dsmiley's [comment on PR 815](https://github.com/apache/solr/pull/815#issuecomment-1106880696):
   
   >I suspect the searcher and/or request being open after the core close will keep the core increment one higher and thus not actually closed until the searcher/request is.
   
   I dug into this a bit more and it's quite tricky.
   
   The problem manifests when the `core.close()` call happens before `searcher.decref()` ... but only if the `core.close()` call in question is the _last_ call to `close()` for the given core. Until `core.refCount` becomes `0`, `core.close()` is basically just bookkeeping; but the _last_ `core.close()` proceeds to `doClose()`, which calls `directoryFactory.close()` -- the `CachingDirectoryFactory` implementation of which blocks waiting for its cached `Directory` instances to be released. The `SolrIndexSearcher` whose closing was reordered to follow `core.close()` actually holds a reference to the cached `Directory` instance, not the `SolrCore` itself, so the thread indeed leaks/resource is not closed.
   
   Looking more closely at why this issue addressed by #815 was hard to reproduce: the bulk of the replication in `TestReplicationHandler` takes place via explicit `fetchindex` command -- almost always with the special test param `wait=true`, causing the associated replication fetch to take place (from start to finish) within the context/lifetime of the enclosing `SolrQueryRequest`, which itself caries a reference to the `SolrCore`, ensuring that the replication code will _not_ be the last call to `core.close()`. So `core.close()` amounts to just a decrement of a refcount -- it doesn't block, and the `searcher.decref()` happens ok, even though out-of-order.
   
   The problem happens when _polling_ replication ("indexFetcher" thread) happens to hit around the time of core close -- all the other core references have been decremented, and the affected code is left holding the only remaining reference to the core _and_ (via the `SolrIndexSearcher`) to the `Directory` instance. Polling replication is vulnerable (whereas explicit `fetchindex` command is not) because polling is a scheduled task that doesn't take place within the context of a `SolrQueryRequest` holding references to the `SolrCore`. Indeed, I verified that the problem addressed by #815 could be equally (though hackily) addressed by wrapping the polling `doFetch()` in a try-with-resources block artificially incrementing the refCount for the SolrCore!
   
   I'm not sure whether there might be other cases that would trigger this, especially with other areas of the code; but I think the prudent thing to do for now would be to take the instances identified in this PR and restore the original close order -- i.e. `SolrIndexSearcher.decref()` (or `SolrQueryRequest.close()`, which in turn may call `SolrIndexSearcher.decref()`) _before_ calling `close()` on the associated `SolrCore`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org