You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2019/09/04 15:58:00 UTC
[jira] [Updated] (FLINK-13963) Consolidate Hadoop file systems
usage and Hadoop integration docs
[ https://issues.apache.org/jira/browse/FLINK-13963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated FLINK-13963:
-----------------------------------
Labels: pull-request-available (was: )
> Consolidate Hadoop file systems usage and Hadoop integration docs
> -----------------------------------------------------------------
>
> Key: FLINK-13963
> URL: https://issues.apache.org/jira/browse/FLINK-13963
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem, Connectors / Hadoop Compatibility, Documentation, FileSystems
> Reporter: Andrey Zagrebin
> Assignee: Andrey Zagrebin
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
>
> We have hadoop related docs in several places at the moment:
> * *dev/batch/connectors.md* (Hadoop FS implementations and setup)
> * *dev/batch/hadoop_compatibility.md* (not valid any more that Flink always has Hadoop types out of the box as we do not build and provide Flink with Hadoop by default)
> * *ops/filesystems/index.md* (plugins, Hadoop FS implementations and setup revisited)
> * *ops/deployment/hadoop.md* (Hadoop classpath)
> * *ops/config.md* (deprecated way to provide Hadoop configuration in Flink conf)
> We could consolidate all these pieces of docs into a consistent structure to help users to navigate through the docs to well-defined spots depending on which feature they are trying to use.
> The places in docs which should contain the information about Hadoop:
> * *dev/batch/hadoop_compatibility.md* (only Dataset API specific stuff about integration with Hadoop)
> * *ops/filesystems/index.md* (Flink FS plugins and Hadoop FS implementations)
> * *ops/deployment/hadoop.md* (Hadoop configuration and classpath)
> How to setup Hadoop itself should be only in *ops/deployment/hadoop.md*. All other places dealing with Hadoop/HDFS should contain only their related things and just reference it 'how to configure Hadoop'. Like all chapters about writing to file systems (batch connectors and streaming file sinks) should just reference file systems.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)