You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by "Jay Sen (Jira)" <ji...@apache.org> on 2019/10/04 01:11:00 UTC
[jira] [Created] (GOBBLIN-900) FsDataWriter dont use writer config
for FileSystem
Jay Sen created GOBBLIN-900:
-------------------------------
Summary: FsDataWriter dont use writer config for FileSystem
Key: GOBBLIN-900
URL: https://issues.apache.org/jira/browse/GOBBLIN-900
Project: Apache Gobblin
Issue Type: Bug
Components: gobblin-core
Affects Versions: 0.14.0
Reporter: Jay Sen
Assignee: Abhishek Tiwari
Fix For: 0.15.0
FsDataWriter.Java:107
{{this.fileContext = FileContext.getFileContext(conf);}}
FileContext is used while moving data from staging to final output path. currently it uses default hadoop Configuration which can be picked up by local hadoop config in classpath, which is not right. the Writer fiulesystem should be always driven by the writer config mentioned as per job or platform config.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)