You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by "Kaepke, Marc" <ma...@haw-hamburg.de> on 2017/08/04 12:13:16 UTC

k8s FileNotFoundException

Hi everyone,

I always get an FileNotFoundException by following the kubernetes setup guide [1].

I moved my jar and my input file onto the job manager pod
After that I join the job manager pod by using: kubectl exec -it <jobmanager> - - /bin/bash

With ls I can see both files. WordCount-example worked well.

Caused by: java.io.IOException: Error opening the Input Split file:/opt/flink/wiki-vote.txt [0,1095061]: /opt/flink/wiki-vote.txt (No such file or directory)
at org.apache.flink.api.common.io.FileInputFormat.open(FileInputFormat.java:706)
at org.apache.flink.api.common.io.DelimitedInputFormat.open(DelimitedInputFormat.java:477)
at org.apache.flink.api.common.io.GenericCsvInputFormat.open(GenericCsvInputFormat.java:301)
at org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:48)
at org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:31)
at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:145)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.FileNotFoundException: /opt/flink/wiki-vote.txt (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.flink.core.fs.local.LocalDataInputStream.<init>(LocalDataInputStream.java:49)
at org.apache.flink.core.fs.local.LocalFileSystem.open(LocalFileSystem.java:141)
at org.apache.flink.api.common.io.FileInputFormat$InputSplitOpenThread.run(FileInputFormat.java:866)
root@flink-jobmanager-2891032829-pqgwq:/opt/flink#

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/kubernetes.html


Any idea?!


Marc

Re: k8s FileNotFoundException

Posted by Aljoscha Krettek <al...@apache.org>.
Is the file also available on the TaskManager containers? These are doing the actual reading, in fact.

In general, I would suggest to use a distributed file system as input for a Flink Job if you want to process files.

Best,
Aljoscha
> On 4. Aug 2017, at 14:13, Kaepke, Marc <ma...@haw-hamburg.de> wrote:
> 
> Hi everyone,
> 
> I always get an FileNotFoundException by following the kubernetes setup guide [1].
> 
> I moved my jar and my input file onto the job manager pod 
> After that I join the job manager pod by using: kubectl exec -it <jobmanager> - - /bin/bash
> 
> With ls I can see both files. WordCount-example worked well.
> 
> Caused by: java.io.IOException: Error opening the Input Split file:/opt/flink/wiki-vote.txt [0,1095061]: /opt/flink/wiki-vote.txt (No such file or directory)
> at org.apache.flink.api.common.io.FileInputFormat.open(FileInputFormat.java:706)
> at org.apache.flink.api.common.io.DelimitedInputFormat.open(DelimitedInputFormat.java:477)
> at org.apache.flink.api.common.io.GenericCsvInputFormat.open(GenericCsvInputFormat.java:301)
> at org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:48)
> at org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:31)
> at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:145)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.FileNotFoundException: /opt/flink/wiki-vote.txt (No such file or directory)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.<init>(FileInputStream.java:138)
> at org.apache.flink.core.fs.local.LocalDataInputStream.<init>(LocalDataInputStream.java:49)
> at org.apache.flink.core.fs.local.LocalFileSystem.open(LocalFileSystem.java:141)
> at org.apache.flink.api.common.io.FileInputFormat$InputSplitOpenThread.run(FileInputFormat.java:866)
> root@flink-jobmanager-2891032829-pqgwq:/opt/flink# 
> 
> [1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/kubernetes.html <https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/kubernetes.html>
> 
> 
> Any idea?!
> 
> 
> Marc