You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "James Ji (JIRA)" <ji...@apache.org> on 2012/10/26 18:05:12 UTC
[jira] [Created] (SOLR-3997) Have solr config and index files on
HDFS.
James Ji created SOLR-3997:
------------------------------
Summary: Have solr config and index files on HDFS.
Key: SOLR-3997
URL: https://issues.apache.org/jira/browse/SOLR-3997
Project: Solr
Issue Type: Wish
Components: SearchComponents - other
Affects Versions: 4.0
Reporter: James Ji
We are currently working on having Solr files read from HDFS. We extended some of the classes so as to avoid modifying the original Solr code and make it compatible with the future release. So here comes the question, I found in QueryElevationComponent, there is a piece of code checking whether elevate.xml exists at local file system. I am wondering if there is a way to by pass this?
QueryElevationComponent.inform(){
....
File fC = new File(core.getResourceLoader().getConfigDir(), f);
File fD = new File(core.getDataDir(), f);
if (fC.exists() == fD.exists()) {
throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
"QueryElevationComponent missing config file: '" + f + "\n"
+ "either: " + fC.getAbsolutePath() + " or " + fD.getAbsolutePath() + " must exist, but not both.");
}
if (fC.exists()) {
exists = true;
log.info("Loading QueryElevation from: "+fC.getAbsolutePath());
Config cfg = new Config(core.getResourceLoader(), f);
elevationCache.put(null, loadElevationMap(cfg));
}
....
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-3997) QueryElevationComponent using native
File object to access file
Posted by "James Ji (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
James Ji updated SOLR-3997:
---------------------------
Description:
We are currently working on having Solr files read from HDFS. We have a class which extends Directory class to handle all file access. However, we found that QueryElevationComponent is not using Directory class but using the native File object.
QueryElevationComponent.inform(){
....
File fC = new File(core.getResourceLoader().getConfigDir(), f);
File fD = new File(core.getDataDir(), f);
if (fC.exists() == fD.exists()) {
throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
"QueryElevationComponent missing config file: '" + f + "\n"
+ "either: " + fC.getAbsolutePath() + " or " + fD.getAbsolutePath() + " must exist, but not both.");
}
if (fC.exists()) {
exists = true;
log.info("Loading QueryElevation from: "+fC.getAbsolutePath());
Config cfg = new Config(core.getResourceLoader(), f);
elevationCache.put(null, loadElevationMap(cfg));
}
....
}
was:
We are currently working on having Solr files read from HDFS. We extended some of the classes so as to avoid modifying the original Solr code and make it compatible with the future release. So here comes the question, I found in QueryElevationComponent, there is a piece of code checking whether elevate.xml exists at local file system. I am wondering if there is a way to by pass this?
QueryElevationComponent.inform(){
....
File fC = new File(core.getResourceLoader().getConfigDir(), f);
File fD = new File(core.getDataDir(), f);
if (fC.exists() == fD.exists()) {
throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
"QueryElevationComponent missing config file: '" + f + "\n"
+ "either: " + fC.getAbsolutePath() + " or " + fD.getAbsolutePath() + " must exist, but not both.");
}
if (fC.exists()) {
exists = true;
log.info("Loading QueryElevation from: "+fC.getAbsolutePath());
Config cfg = new Config(core.getResourceLoader(), f);
elevationCache.put(null, loadElevationMap(cfg));
}
....
}
Issue Type: Bug (was: Wish)
Summary: QueryElevationComponent using native File object to access file (was: Have solr config and index files on HDFS.)
> QueryElevationComponent using native File object to access file
> ---------------------------------------------------------------
>
> Key: SOLR-3997
> URL: https://issues.apache.org/jira/browse/SOLR-3997
> Project: Solr
> Issue Type: Bug
> Components: SearchComponents - other
> Affects Versions: 4.0
> Reporter: James Ji
>
> We are currently working on having Solr files read from HDFS. We have a class which extends Directory class to handle all file access. However, we found that QueryElevationComponent is not using Directory class but using the native File object.
> QueryElevationComponent.inform(){
> ....
> File fC = new File(core.getResourceLoader().getConfigDir(), f);
> File fD = new File(core.getDataDir(), f);
> if (fC.exists() == fD.exists()) {
> throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
> "QueryElevationComponent missing config file: '" + f + "\n"
> + "either: " + fC.getAbsolutePath() + " or " + fD.getAbsolutePath() + " must exist, but not both.");
> }
> if (fC.exists()) {
> exists = true;
> log.info("Loading QueryElevation from: "+fC.getAbsolutePath());
> Config cfg = new Config(core.getResourceLoader(), f);
> elevationCache.put(null, loadElevationMap(cfg));
> }
> ....
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3997) Have solr config and index files on
HDFS.
Posted by "Otis Gospodnetic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485234#comment-13485234 ]
Otis Gospodnetic commented on SOLR-3997:
----------------------------------------
James - try asking on the mailing list, not in JIRA.
> Have solr config and index files on HDFS.
> -----------------------------------------
>
> Key: SOLR-3997
> URL: https://issues.apache.org/jira/browse/SOLR-3997
> Project: Solr
> Issue Type: Wish
> Components: SearchComponents - other
> Affects Versions: 4.0
> Reporter: James Ji
>
> We are currently working on having Solr files read from HDFS. We extended some of the classes so as to avoid modifying the original Solr code and make it compatible with the future release. So here comes the question, I found in QueryElevationComponent, there is a piece of code checking whether elevate.xml exists at local file system. I am wondering if there is a way to by pass this?
> QueryElevationComponent.inform(){
> ....
> File fC = new File(core.getResourceLoader().getConfigDir(), f);
> File fD = new File(core.getDataDir(), f);
> if (fC.exists() == fD.exists()) {
> throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
> "QueryElevationComponent missing config file: '" + f + "\n"
> + "either: " + fC.getAbsolutePath() + " or " + fD.getAbsolutePath() + " must exist, but not both.");
> }
> if (fC.exists()) {
> exists = true;
> log.info("Loading QueryElevation from: "+fC.getAbsolutePath());
> Config cfg = new Config(core.getResourceLoader(), f);
> elevationCache.put(null, loadElevationMap(cfg));
> }
> ....
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org