You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Soheil Pourbafrani <so...@gmail.com> on 2020/01/15 09:58:09 UTC

How Flink read files from local filesystem

Hi,

Suppose we have a Flink single node cluster with multiple slots and some
input files exist in local file system. In this case where we have no
distributed file system to dedicate each file's block to taskmanagers, how
Flink will read the file? Do all the task managers will open the file
separately and read some dedicated portion of the file in parallel?

Re: How Flink read files from local filesystem

Posted by Tillman Peng <yh...@orange.fr>.

You can use env.readTextFile(path) which accepts path to a directory and 
reads all files in that directory producing record for each line.

on 2020/1/15 17:58, Soheil Pourbafrani wrote:
> Suppose we have a Flink single node cluster with multiple slots and some 
> input files exist in local file system. In this case where we have no 
> distributed file system to dedicate each file's block to taskmanagers, 
> how Flink will read the file? Do all the task managers will open the 
> file separately and read some dedicated portion of the file in parallel?