You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by Jay Sen <ja...@apache.org> on 2019/05/11 01:00:04 UTC
Hive source is file based extractor ?
Hi Gobblin Team,
I notice that to run hive to hive copy, even when job config mentions
source and target hive metastore details, it does not work until you
explicitly specify source.filebased.fs.uri ( for source) and writer.fs.uri
(for target ) to point to hdfs://
the above are File Extractor based propeties, wonder why its used while
trying to figure out source and target FileSystem in CopySource.
final FileSystem sourceFs = HadoopUtils.getSourceFileSystem(state);
final FileSystem targetFs = HadoopUtils.getWriterFileSystem(state, 1,
0);
wont the source and target hive metastore enough to say its hdfs source and
target ?
Thanks
Jay