You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Radu Gheorghe (JIRA)" <ji...@apache.org> on 2017/10/12 10:30:00 UTC
[jira] [Created] (SOLR-11473) Make HDFSDirectoryFactory support
other prefixes (besides hdfs:/)
Radu Gheorghe created SOLR-11473:
------------------------------------
Summary: Make HDFSDirectoryFactory support other prefixes (besides hdfs:/)
Key: SOLR-11473
URL: https://issues.apache.org/jira/browse/SOLR-11473
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: hdfs
Affects Versions: 6.6.1
Reporter: Radu Gheorghe
Priority: Minor
Not sure if it's a bug or a missing feature :) I'm trying to make Solr work on Alluxio, as described by [~thelabdude] in https://www.slideshare.net/thelabdude/running-solr-in-the-cloud-at-memory-speed-with-alluxio/1
The problem I'm facing here is with autoAddReplicas. If I have replicationFactor=1 and the node with that replica dies, the node taking over incorrectly assigns the data directory. For example:
before
{code}"dataDir":"alluxio://localhost:19998/solr/test/",{code}
after
{code}"dataDir":"alluxio://localhost:19998/solr/test/core_node1/alluxio://localhost:19998/solr/test/",{code}
The same happens for ulogDir. Apparently, this has to do with this bit from HDFSDirectoryFactory:
{code} public boolean isAbsolute(String path) {
return path.startsWith("hdfs:/");
}{code}
If I add "alluxio:/" in there, the paths are correct and the index is recovered.
I see a few options here:
* add "alluxio:/" to the list there
* add a regular expression in the lines of \[a-z]*:/ I hope that's not too expensive, I'm not sure how often this method is called
* don't do anything and expect alluxio to work with an "hdfs:/" path? I actually tried that and didn't manage to make it work
* have a different DirectoryFactory or something else?
What do you think?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org