You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Kevin Risden (JIRA)" <ji...@apache.org> on 2019/03/26 20:32:00 UTC
[jira] [Resolved] (SOLR-11335) HdfsDirectory & Factory should not
close the FileSystem object retrieved with get
[ https://issues.apache.org/jira/browse/SOLR-11335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Risden resolved SOLR-11335.
---------------------------------
Resolution: Workaround
Marking this as resolved with workaround. SOLR-11473 would be the long term fix. Short term fix is to make sure that "fs.SCHEME.impl.disable.cache=true" for whatever filesystem you are using if it is NOT HDFS.
> HdfsDirectory & Factory should not close the FileSystem object retrieved with get
> ---------------------------------------------------------------------------------
>
> Key: SOLR-11335
> URL: https://issues.apache.org/jira/browse/SOLR-11335
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Hadoop Integration, hdfs
> Reporter: Timothy Potter
> Priority: Minor
>
> I'm seeing issues where the Hadoop FileSystem instance is closed out from under other objects. From what I understand, the Hadoop FileSystem object (org.apache.hadoop.fs.FileSystem) retrieved with {{FileSystem.get}} as is done in HdfsDirectory's ctor is a shared object that if closed, can affect other code using that same shared instance. You can see this is a cached, shared object here ->
> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java#L455
> Thus, I suspect Solr should not be closing any FileSystem instance retrieved with get. It's important to mention that if I set the {{fs.$SCHEME.impl.disable.cache}} to true, then my problems go away, which seems to confirm that Solr is using the API incorrectly. That being said, I'm surprised this hasn't been raised before, so maybe I've missed something basic in Solr's use of HDFS?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org