[jira] [Created] (TINKERPOP-1331) HADOOP_GREMLIN_LIBS can only point to local file system

["Daniel Kuppitz (JIRA)" <ji...@apache.org>]

Daniel Kuppitz created TINKERPOP-1331:
-----------------------------------------

             Summary: HADOOP_GREMLIN_LIBS can only point to local file system
                 Key: TINKERPOP-1331
                 URL: https://issues.apache.org/jira/browse/TINKERPOP-1331
             Project: TinkerPop
          Issue Type: Improvement
          Components: hadoop
    Affects Versions: 3.1.2-incubating, 3.2.0-incubating
            Reporter: Daniel Kuppitz


These two lines in {{SparkGraphComputer}} assume that {{HADOOP_GREMLIN_LIBS}} will only contain local file system references (although it seems that the rest of the code could handle DFS references):

{code}
final String[] paths = hadoopGremlinLocalLibs.split(":");
final FileSystem fs = FileSystem.get(hadoopConfiguration);
{code}

If, for example, {{HADOOP_GREMLIN_LIBS}} would be set to {{hdfs:///spark-gremlin-libs:/foo/bar}}, the {{split(":")}} call would obviously separate the file system scheme ({{hdfs://}}) from the path ({{/spark-gremlin-libs}}).

Next, {{FileSystem.get(hadoopConfiguration)}} will always only return a reference to the {{FileSystem}} that is defined as the default file system.

The same is probably true for {{GiraphGraphComputer}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)