[jira] [Commented] (SOLR-14653) Replace usages of org.apache.hadoop.fs.Path that are used to construct Zookeeper paths

["Uwe Schindler (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-14653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159325#comment-17159325 ] 

Uwe Schindler commented on SOLR-14653:
--------------------------------------

Thanks for opening this issue! Indeed we should try to use path-modifying methods coming from zookeeper client library directly. Maybe they were added since Mark Miller implemented this back in time.

{{java.net.URI}} would be my fallback, the above example with leaderPath could be written as (untested):

{code:java}
String parent = new URI(null, null, leaderPath, null).resolve("..").getPath();
{code}

> Replace usages of org.apache.hadoop.fs.Path that are used to construct Zookeeper paths
> --------------------------------------------------------------------------------------
>
>                 Key: SOLR-14653
>                 URL: https://issues.apache.org/jira/browse/SOLR-14653
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Erick Erickson
>            Assignee: Erick Erickson
>            Priority: Major
>
> See SOLR-13939. We construct Zookeeper paths at least in ShardLeaderElectionContextBase.java by using methods in org.apache.hadoop.fs.Path e.g. "String parent = new Path(leaderPath).getParent().toString()". This works because  the Hadoop method uses forward slashes even on Windows machines.
> If those calls are changed to use nio Paths.get or similar, we can't run on Windows. Relying on hdfs classes to construct these paths in code unrelated to HDFS is suspicious at best.
> We should examine all inclusions of org.apache.hadoop.fs.Path and replace them with something else. Uwe suggests java.net.URI as one possibility.
> We should check to see what Zookeeper provides.
> There are 5 files that import org.apache.hadoop... that do _not_ have HDFS in their name, and 4 of those are probably legitimate (SolrSnapshotsTool for instance) so checking all of them won't take long.
> From Uwe's comment in SOLR-13939:
> {quote}"Hi,
>  Windows is happy again. So your fix helped. I responded to mailing list, here again:
> In Zookeeper all paths are forward slashes (as it is a "path" inside the zookeeper environment). If you replace that with a NIO Windows Path it creates backslashes. This makes Zookeeper fail (of course).
> Actually using the Hadoop class is broken workaround, it may be better to use the more abstract java.net.URI class for correct "path" transformations. The risk with java.nio.file.Path is also that if the path exists in file system it handles the resolving different. So if some user has for some reason a local path exactly looking like a zookeper node path it would use different behaviour.
> So my suggestion: Use java.net.URI to build the path and use URI.getPath() or similar to make a String out of it. The alternative to replace all backslashes in the path to forwards slashes is a bad idea for the reasons I explained before.
> Now we should replace the hadoop class IMHO by java.net.URI.
> {quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org