You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by "Jay Sen (Jira)" <ji...@apache.org> on 2019/10/04 01:11:00 UTC

[jira] [Created] (GOBBLIN-900) FsDataWriter dont use writer config for FileSystem

Jay Sen created GOBBLIN-900:
-------------------------------

             Summary: FsDataWriter dont use writer config for FileSystem
                 Key: GOBBLIN-900
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-900
             Project: Apache Gobblin
          Issue Type: Bug
          Components: gobblin-core
    Affects Versions: 0.14.0
            Reporter: Jay Sen
            Assignee: Abhishek Tiwari
             Fix For: 0.15.0


FsDataWriter.Java:107
{{this.fileContext = FileContext.getFileContext(conf);}} 

FileContext is used while moving data from staging to final output path. currently it uses default hadoop Configuration which can be picked up by local hadoop config in classpath, which is not right. the Writer fiulesystem should be always driven by the writer config mentioned as per job or platform config.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)