You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Madabhattula Rajesh Kumar <mr...@gmail.com> on 2015/07/19 16:36:11 UTC
Run Time Exception
Hi,
I have written simple wordcount program in scala. When I execute the
program, I'm getting below exception.
Please let me know how to fix this issue. I'm using Flink 0.9.0 version
*Below is the program :-*
val env = ExecutionEnvironment.getExecutionEnvironment
// get input data
val text = env readTextFile("/Users/hadoop2/Data/word.txt")
val counts = text flatMap(l=>l split(" ")) map(word=>(word,1))
groupBy(0) sum(1)
// emit result
counts print
env.execute("TEST")
*Exception :-*
Exception in thread "main" java.lang.RuntimeException: No new data sinks
have been defined since the last execution. The last execution refers to
the latest call to 'execute()', 'count()', 'collect()', or 'print()'.
at
org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:910)
at
org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:893)
at
org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:50)
at
org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:590)
at WordCount$.main(WordCount.scala:17)
at WordCount.main(WordCount.scala)
Regards,
Rajesh
Re: Run Time Exception
Posted by marzieh ghasemi <m....@gmail.com>.
Thank you!
On Thu, Nov 15, 2018 at 4:18 PM Chesnay Schepler <ch...@apache.org> wrote:
> I'm highly encouraging you to read through the examples and Batch API
> documentation:
>
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/batch/#example-program
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/batch/dataset_transformations.html#dataset-transformations
>
> TL;DR: apply a map function to the transactions DataSet
>
> On 15.11.2018 13:42, Mar_zieh wrote:
> > Hello guys
> >
> > I have wrote this code. I could print the whole file, but I want to read
> the
> > file line by line and process each line separately. would you please help
> > how I can do that?
> >
> > ExecutionEnvironment env =
> ExecutionEnvironment.createLocalEnvironment();
> > DataSet<String> transactions =
> > env.readTextFile("/home/cfms11/Downloads/Data.csv");
> > transactions.print();
> >
> > Thank you in advance.
> >
> >
> >
> > --
> > Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
> >
>
>
Re: Run Time Exception
Posted by Chesnay Schepler <ch...@apache.org>.
I'm highly encouraging you to read through the examples and Batch API
documentation:
https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/batch/#example-program
https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/batch/dataset_transformations.html#dataset-transformations
TL;DR: apply a map function to the transactions DataSet
On 15.11.2018 13:42, Mar_zieh wrote:
> Hello guys
>
> I have wrote this code. I could print the whole file, but I want to read the
> file line by line and process each line separately. would you please help
> how I can do that?
>
> ExecutionEnvironment env = ExecutionEnvironment.createLocalEnvironment();
> DataSet<String> transactions =
> env.readTextFile("/home/cfms11/Downloads/Data.csv");
> transactions.print();
>
> Thank you in advance.
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>
Re: Run Time Exception
Posted by Mar_zieh <m....@gmail.com>.
Hello guys
I have wrote this code. I could print the whole file, but I want to read the
file line by line and process each line separately. would you please help
how I can do that?
ExecutionEnvironment env = ExecutionEnvironment.createLocalEnvironment();
DataSet<String> transactions =
env.readTextFile("/home/cfms11/Downloads/Data.csv");
transactions.print();
Thank you in advance.
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Run Time Exception
Posted by Mar_zieh <m....@gmail.com>.
Hello guys
I have wrote this code. I could print the whole file, but I want to read the
file line by line and process each line separately. would you please help
how I can do that?
ExecutionEnvironment env = ExecutionEnvironment.createLocalEnvironment();
DataSet<String> transactions =
env.readTextFile("/home/cfms11/Downloads/Data.csv");
transactions.print();
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Run Time Exception
Posted by Ufuk Celebi <uc...@apache.org>.
So what you should do is just call print and remove the execute call.
– Ufuk
On Sunday, July 19, 2015, Madabhattula Rajesh Kumar <mr...@gmail.com>
wrote:
> Hi Chiwan Park,
>
> Thank you for the clarification
>
> Regards,
> Rajesh
>
> On Sun, Jul 19, 2015 at 11:49 PM, Chiwan Park <chiwanpark@apache.org
> <javascript:_e(%7B%7D,'cvml','chiwanpark@apache.org');>> wrote:
>
>> Hi,
>>
>> Flink program should have at least one data sink. When your program calls
>> `print` method, the method adds a data sink into your program automatically
>> and execute it immediately. If you want to run Flink program without
>> calling `print` method, you should add a data sink into your program and
>> execute it manually by calling `execute` method of `ExecutionEnvironment`
>> object.
>>
>> Note that only some methods about data sink such as `print`, `count` and
>> `collect` execute the program immediately.
>>
>> There are more detail descriptions about data sink [1] and lazy
>> evaluation [2] in Flink documentation. The documentation will help you to
>> understand the structure of Flink program.
>>
>> Regards,
>> Chiwan Park
>>
>> [1]
>> https://ci.apache.org/projects/flink/flink-docs-release-0.9/apis/programming_guide.html#data-sinks
>> [2]
>> https://ci.apache.org/projects/flink/flink-docs-release-0.9/apis/programming_guide.html#lazy-evaluation
>>
>> > On Jul 20, 2015, at 2:59 AM, Madabhattula Rajesh Kumar <
>> mrajaforu@gmail.com <javascript:_e(%7B%7D,'cvml','mrajaforu@gmail.com');>>
>> wrote:
>> >
>> > Hi Scahin,
>> >
>> > Thank you for the response. I have commented counts print line. After
>> that I got below exception
>> >
>> > Exception in thread "main" java.lang.RuntimeException: No data sinks
>> have been created yet. A program needs at least one sink that consumes
>> data. Examples are writing the data set or printing it.
>> > at
>> org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:914)
>> > at
>> org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:893)
>> > at
>> org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:50)
>> > at
>> org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:590)
>> > at WordCount$.main(WordCount.scala:13)
>> > at WordCount.main(WordCount.scala)
>> >
>> > Regards,
>> > Rajesh
>> >
>> > On Sun, Jul 19, 2015 at 8:26 PM, Sachin Goel <sachingoel0101@gmail.com
>> <javascript:_e(%7B%7D,'cvml','sachingoel0101@gmail.com');>> wrote:
>> > Hi
>> > You do not need to call env.execute after doing a print call. Print
>> itself triggers the execution. The reason for the Exception is quite
>> obvious. After the print call, there is no sink for the program execution.
>> So, execution cannot proceed.
>> > You can however explicitly define a sink and then call env.execute.
>> >
>> > Cheers!
>> > Sachin
>> >
>> > -- Sachin Goel
>> > Computer Science, IIT Delhi
>> > m. +91-9871457685
>> >
>> > On Sun, Jul 19, 2015 at 8:06 PM, Madabhattula Rajesh Kumar <
>> mrajaforu@gmail.com <javascript:_e(%7B%7D,'cvml','mrajaforu@gmail.com');>>
>> wrote:
>> > Hi,
>> >
>> > I have written simple wordcount program in scala. When I execute the
>> program, I'm getting below exception.
>> >
>> > Please let me know how to fix this issue. I'm using Flink 0.9.0 version
>> >
>> > Below is the program :-
>> >
>> > val env = ExecutionEnvironment.getExecutionEnvironment
>> > // get input data
>> > val text = env readTextFile("/Users/hadoop2/Data/word.txt")
>> > val counts = text flatMap(l=>l split(" ")) map(word=>(word,1))
>> groupBy(0) sum(1)
>> > // emit result
>> > counts print
>> > env.execute("TEST")
>> >
>> > Exception :-
>> >
>> > Exception in thread "main" java.lang.RuntimeException: No new data
>> sinks have been defined since the last execution. The last execution refers
>> to the latest call to 'execute()', 'count()', 'collect()', or 'print()'.
>> > at
>> org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:910)
>> > at
>> org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:893)
>> > at
>> org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:50)
>> > at
>> org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:590)
>> > at WordCount$.main(WordCount.scala:17)
>> > at WordCount.main(WordCount.scala)
>> >
>> > Regards,
>> > Rajesh
>> >
>> >
>>
>>
>>
>>
>>
>>
>
Re: Run Time Exception
Posted by Madabhattula Rajesh Kumar <mr...@gmail.com>.
Hi Chiwan Park,
Thank you for the clarification
Regards,
Rajesh
On Sun, Jul 19, 2015 at 11:49 PM, Chiwan Park <ch...@apache.org> wrote:
> Hi,
>
> Flink program should have at least one data sink. When your program calls
> `print` method, the method adds a data sink into your program automatically
> and execute it immediately. If you want to run Flink program without
> calling `print` method, you should add a data sink into your program and
> execute it manually by calling `execute` method of `ExecutionEnvironment`
> object.
>
> Note that only some methods about data sink such as `print`, `count` and
> `collect` execute the program immediately.
>
> There are more detail descriptions about data sink [1] and lazy evaluation
> [2] in Flink documentation. The documentation will help you to understand
> the structure of Flink program.
>
> Regards,
> Chiwan Park
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-0.9/apis/programming_guide.html#data-sinks
> [2]
> https://ci.apache.org/projects/flink/flink-docs-release-0.9/apis/programming_guide.html#lazy-evaluation
>
> > On Jul 20, 2015, at 2:59 AM, Madabhattula Rajesh Kumar <
> mrajaforu@gmail.com> wrote:
> >
> > Hi Scahin,
> >
> > Thank you for the response. I have commented counts print line. After
> that I got below exception
> >
> > Exception in thread "main" java.lang.RuntimeException: No data sinks
> have been created yet. A program needs at least one sink that consumes
> data. Examples are writing the data set or printing it.
> > at
> org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:914)
> > at
> org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:893)
> > at
> org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:50)
> > at
> org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:590)
> > at WordCount$.main(WordCount.scala:13)
> > at WordCount.main(WordCount.scala)
> >
> > Regards,
> > Rajesh
> >
> > On Sun, Jul 19, 2015 at 8:26 PM, Sachin Goel <sa...@gmail.com>
> wrote:
> > Hi
> > You do not need to call env.execute after doing a print call. Print
> itself triggers the execution. The reason for the Exception is quite
> obvious. After the print call, there is no sink for the program execution.
> So, execution cannot proceed.
> > You can however explicitly define a sink and then call env.execute.
> >
> > Cheers!
> > Sachin
> >
> > -- Sachin Goel
> > Computer Science, IIT Delhi
> > m. +91-9871457685
> >
> > On Sun, Jul 19, 2015 at 8:06 PM, Madabhattula Rajesh Kumar <
> mrajaforu@gmail.com> wrote:
> > Hi,
> >
> > I have written simple wordcount program in scala. When I execute the
> program, I'm getting below exception.
> >
> > Please let me know how to fix this issue. I'm using Flink 0.9.0 version
> >
> > Below is the program :-
> >
> > val env = ExecutionEnvironment.getExecutionEnvironment
> > // get input data
> > val text = env readTextFile("/Users/hadoop2/Data/word.txt")
> > val counts = text flatMap(l=>l split(" ")) map(word=>(word,1))
> groupBy(0) sum(1)
> > // emit result
> > counts print
> > env.execute("TEST")
> >
> > Exception :-
> >
> > Exception in thread "main" java.lang.RuntimeException: No new data sinks
> have been defined since the last execution. The last execution refers to
> the latest call to 'execute()', 'count()', 'collect()', or 'print()'.
> > at
> org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:910)
> > at
> org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:893)
> > at
> org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:50)
> > at
> org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:590)
> > at WordCount$.main(WordCount.scala:17)
> > at WordCount.main(WordCount.scala)
> >
> > Regards,
> > Rajesh
> >
> >
>
>
>
>
>
>
Re: Run Time Exception
Posted by Chiwan Park <ch...@apache.org>.
Hi,
Flink program should have at least one data sink. When your program calls `print` method, the method adds a data sink into your program automatically and execute it immediately. If you want to run Flink program without calling `print` method, you should add a data sink into your program and execute it manually by calling `execute` method of `ExecutionEnvironment` object.
Note that only some methods about data sink such as `print`, `count` and `collect` execute the program immediately.
There are more detail descriptions about data sink [1] and lazy evaluation [2] in Flink documentation. The documentation will help you to understand the structure of Flink program.
Regards,
Chiwan Park
[1] https://ci.apache.org/projects/flink/flink-docs-release-0.9/apis/programming_guide.html#data-sinks
[2] https://ci.apache.org/projects/flink/flink-docs-release-0.9/apis/programming_guide.html#lazy-evaluation
> On Jul 20, 2015, at 2:59 AM, Madabhattula Rajesh Kumar <mr...@gmail.com> wrote:
>
> Hi Scahin,
>
> Thank you for the response. I have commented counts print line. After that I got below exception
>
> Exception in thread "main" java.lang.RuntimeException: No data sinks have been created yet. A program needs at least one sink that consumes data. Examples are writing the data set or printing it.
> at org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:914)
> at org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:893)
> at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:50)
> at org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:590)
> at WordCount$.main(WordCount.scala:13)
> at WordCount.main(WordCount.scala)
>
> Regards,
> Rajesh
>
> On Sun, Jul 19, 2015 at 8:26 PM, Sachin Goel <sa...@gmail.com> wrote:
> Hi
> You do not need to call env.execute after doing a print call. Print itself triggers the execution. The reason for the Exception is quite obvious. After the print call, there is no sink for the program execution. So, execution cannot proceed.
> You can however explicitly define a sink and then call env.execute.
>
> Cheers!
> Sachin
>
> -- Sachin Goel
> Computer Science, IIT Delhi
> m. +91-9871457685
>
> On Sun, Jul 19, 2015 at 8:06 PM, Madabhattula Rajesh Kumar <mr...@gmail.com> wrote:
> Hi,
>
> I have written simple wordcount program in scala. When I execute the program, I'm getting below exception.
>
> Please let me know how to fix this issue. I'm using Flink 0.9.0 version
>
> Below is the program :-
>
> val env = ExecutionEnvironment.getExecutionEnvironment
> // get input data
> val text = env readTextFile("/Users/hadoop2/Data/word.txt")
> val counts = text flatMap(l=>l split(" ")) map(word=>(word,1)) groupBy(0) sum(1)
> // emit result
> counts print
> env.execute("TEST")
>
> Exception :-
>
> Exception in thread "main" java.lang.RuntimeException: No new data sinks have been defined since the last execution. The last execution refers to the latest call to 'execute()', 'count()', 'collect()', or 'print()'.
> at org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:910)
> at org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:893)
> at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:50)
> at org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:590)
> at WordCount$.main(WordCount.scala:17)
> at WordCount.main(WordCount.scala)
>
> Regards,
> Rajesh
>
>
Re: Run Time Exception
Posted by Madabhattula Rajesh Kumar <mr...@gmail.com>.
Hi Scahin,
Thank you for the response. I have commented *counts print *line. After
that I got below exception
Exception in thread "main" java.lang.RuntimeException: No data sinks have
been created yet. A program needs at least one sink that consumes data.
Examples are writing the data set or printing it.
at
org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:914)
at
org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:893)
at
org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:50)
at
org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:590)
at WordCount$.main(WordCount.scala:13)
at WordCount.main(WordCount.scala)
Regards,
Rajesh
On Sun, Jul 19, 2015 at 8:26 PM, Sachin Goel <sa...@gmail.com>
wrote:
> Hi
> You do not need to call env.execute after doing a print call. Print itself
> triggers the execution. The reason for the Exception is quite obvious.
> After the print call, there is no sink for the program execution. So,
> execution cannot proceed.
> You can however explicitly define a sink and then call env.execute.
>
> Cheers!
> Sachin
>
> -- Sachin Goel
> Computer Science, IIT Delhi
> m. +91-9871457685
>
> On Sun, Jul 19, 2015 at 8:06 PM, Madabhattula Rajesh Kumar <
> mrajaforu@gmail.com> wrote:
>
>> Hi,
>>
>> I have written simple wordcount program in scala. When I execute the
>> program, I'm getting below exception.
>>
>> Please let me know how to fix this issue. I'm using Flink 0.9.0 version
>>
>> *Below is the program :-*
>>
>> val env = ExecutionEnvironment.getExecutionEnvironment
>> // get input data
>> val text = env readTextFile("/Users/hadoop2/Data/word.txt")
>> val counts = text flatMap(l=>l split(" ")) map(word=>(word,1))
>> groupBy(0) sum(1)
>> // emit result
>> counts print
>> env.execute("TEST")
>>
>> *Exception :-*
>>
>> Exception in thread "main" java.lang.RuntimeException: No new data sinks
>> have been defined since the last execution. The last execution refers to
>> the latest call to 'execute()', 'count()', 'collect()', or 'print()'.
>> at
>> org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:910)
>> at
>> org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:893)
>> at
>> org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:50)
>> at
>> org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:590)
>> at WordCount$.main(WordCount.scala:17)
>> at WordCount.main(WordCount.scala)
>>
>> Regards,
>> Rajesh
>>
>
>
Re: Run Time Exception
Posted by Sachin Goel <sa...@gmail.com>.
Hi
You do not need to call env.execute after doing a print call. Print itself
triggers the execution. The reason for the Exception is quite obvious.
After the print call, there is no sink for the program execution. So,
execution cannot proceed.
You can however explicitly define a sink and then call env.execute.
Cheers!
Sachin
-- Sachin Goel
Computer Science, IIT Delhi
m. +91-9871457685
On Sun, Jul 19, 2015 at 8:06 PM, Madabhattula Rajesh Kumar <
mrajaforu@gmail.com> wrote:
> Hi,
>
> I have written simple wordcount program in scala. When I execute the
> program, I'm getting below exception.
>
> Please let me know how to fix this issue. I'm using Flink 0.9.0 version
>
> *Below is the program :-*
>
> val env = ExecutionEnvironment.getExecutionEnvironment
> // get input data
> val text = env readTextFile("/Users/hadoop2/Data/word.txt")
> val counts = text flatMap(l=>l split(" ")) map(word=>(word,1))
> groupBy(0) sum(1)
> // emit result
> counts print
> env.execute("TEST")
>
> *Exception :-*
>
> Exception in thread "main" java.lang.RuntimeException: No new data sinks
> have been defined since the last execution. The last execution refers to
> the latest call to 'execute()', 'count()', 'collect()', or 'print()'.
> at
> org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:910)
> at
> org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:893)
> at
> org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:50)
> at
> org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:590)
> at WordCount$.main(WordCount.scala:17)
> at WordCount.main(WordCount.scala)
>
> Regards,
> Rajesh
>