You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Stefan Bunk <st...@googlemail.com> on 2015/04/14 16:49:20 UTC
Logging in Flink 0.9.0-milestone-1
Hi Flinkers,
I just switched to 0.9.0-milestone-1, and now I get Flink's logging output
again in my console (local execution).
I have a log4j.xml under src/main/resources, which says not to log any
Flink job progress updates, and which worked fine so far:
[...]
<logger name="org.apache.flink">
<level value="ERROR" />
<appender-ref ref="Console" />
</logger>
[...]
See the full file at [1].
However, this file seems to be ignored now. What do I need to change?
Cheers
Stefan
[1] https://gist.github.com/knub/bb1b3ffc6b47e2a1376d
Re: Logging in Flink 0.9.0-milestone-1
Posted by Till Rohrmann <tr...@apache.org>.
Hi Stefan,
You can do this if you disableSysoutLogging and change your
log4j-cli.properties file to also print to console. There you can then
control what is logged to console. However, I think that you have to set
disableSysoutLogging in your program.
Cheers,
Till
On Fri, Jun 26, 2015 at 11:47 AM, Stefan Bunk <st...@googlemail.com>
wrote:
> Hi Robert,
>
> this problem persists in the 0.9 release. Using `disableSysoutLogging()`
> works, but I'd rather configure this in the log4j.xml. Is this possible?
>
> Best,
> Stefan
>
> On 14 April 2015 at 20:55, Robert Metzger <rm...@apache.org> wrote:
>
>> You can control the logging behavior from the ExecutionConfig
>> (env.getExecutionConfig()).
>> There is a method (disableSysoutLogging()) that you can use. (In
>> 0.9-SNAPSHOT only).
>>
>> Sorry again that you ran into this issue.
>>
>> On Tue, Apr 14, 2015 at 8:45 PM, Robert Metzger <rm...@apache.org>
>> wrote:
>>
>>> Ah, I see.
>>>
>>> The issue is this line in the JobClient.scala here:
>>> https://github.com/apache/flink/blob/release-0.9.0-milestone-1-rc1/flink-runtime/src/main/scala/org/apache/flink/runtime/client/JobClient.scala#L97
>>>
>>> As you can see, its doing sysout logging.
>>> In the current master, this has been reworked:
>>> https://github.com/apache/flink/blob/69a400fadd258fe0a1ff0b5670a3611fda4c1cdf/flink-runtime/src/main/java/org/apache/flink/runtime/client/JobClientActor.java#L60
>>> (I'm currently looking up how to control the behavior)
>>>
>>> If its possible for you, you can switch to 0.9-SNAPSHOT.
>>>
>>> Best,
>>> Robert
>>>
>>>
>>> On Tue, Apr 14, 2015 at 7:02 PM, Stefan Bunk <stefan.bunk@googlemail.com
>>> > wrote:
>>>
>>>> Hi,
>>>>
>>>> I am also using IntelliJ, and I am starting directly from the IDE.
>>>> Local execution.
>>>> This is what my logging output looks like: [1].
>>>>
>>>> I am getting my logger via:
>>>> val log = org.apache.log4j.Logger.getLogger(getClass)
>>>>
>>>> [1] https://gist.github.com/knub/1c11683601b4eeb5d51b
>>>>
>>>> On 14 April 2015 at 18:47, Robert Metzger <rm...@apache.org> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> how are you starting Flink? Out of the IDE? Using the scripts?
>>>>>
>>>>> I just created a new flink project with the milestone version. Just
>>>>> putting your log4j.xml into the resources folder enabled the logging (I've
>>>>> set it to INFO for flink and it worked).
>>>>> I've used IntelliJ and started the WordCount.java from the quickstart.
>>>>>
>>>>> I think I need more details about your setup to reproduce the issue.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Apr 14, 2015 at 5:52 PM, Stefan Bunk <
>>>>> stefan.bunk@googlemail.com> wrote:
>>>>>
>>>>>> Hi Robert,
>>>>>>
>>>>>> thanks for the info.
>>>>>> Adding the parameter didn't help. My logging file is found and my
>>>>>> logging configuration for my own logging is working (even without the
>>>>>> parameter), it's just that the file in the jar seems to be preferred over
>>>>>> my file.
>>>>>>
>>>>>> Best,
>>>>>> Stefan
>>>>>>
>>>>>> On 14 April 2015 at 17:16, Robert Metzger <rm...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Stefan,
>>>>>>>
>>>>>>> we made a stupid mistake in the 0.9.0-milestone-1 release by
>>>>>>> including our log4j.properties into the flink-runtime jar. its also in the
>>>>>>> fat jar in flink-dist.
>>>>>>>
>>>>>>> Maybe you can pass the name of your log4j file to your application
>>>>>>> with -Dlog4j.configuration=log4j.xml?
>>>>>>>
>>>>>>> The issue is already fixed in the current master but that doesn't
>>>>>>> help 0.9.0-milestone-1 users :(
>>>>>>>
>>>>>>> Best,
>>>>>>> Robert
>>>>>>>
>>>>>>> On Tue, Apr 14, 2015 at 4:49 PM, Stefan Bunk <
>>>>>>> stefan.bunk@googlemail.com> wrote:
>>>>>>>
>>>>>>>> Hi Flinkers,
>>>>>>>>
>>>>>>>> I just switched to 0.9.0-milestone-1, and now I get Flink's logging
>>>>>>>> output again in my console (local execution).
>>>>>>>> I have a log4j.xml under src/main/resources, which says not to log
>>>>>>>> any Flink job progress updates, and which worked fine so far:
>>>>>>>>
>>>>>>>> [...]
>>>>>>>> <logger name="org.apache.flink">
>>>>>>>> <level value="ERROR" />
>>>>>>>> <appender-ref ref="Console" />
>>>>>>>> </logger>
>>>>>>>> [...]
>>>>>>>> See the full file at [1].
>>>>>>>>
>>>>>>>> However, this file seems to be ignored now. What do I need to
>>>>>>>> change?
>>>>>>>>
>>>>>>>> Cheers
>>>>>>>> Stefan
>>>>>>>>
>>>>>>>> [1] https://gist.github.com/knub/bb1b3ffc6b47e2a1376d
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Logging in Flink 0.9.0-milestone-1
Posted by Stephan Ewen <se...@apache.org>.
There are two different forms of progress logging:
- The log4j framework logs everything (as configured in the
log4j.properties)
- Additionally, the client sysout prints the progress reports
If you want the command line to be quiet, you need to disable sysout
logging on the execution environment for now.
We are thinking about adding a flag to the CLI frontend to make the console
quiet (no progress reports).
Greetings,
Stephan
On Fri, Jun 26, 2015 at 11:47 AM, Stefan Bunk <st...@googlemail.com>
wrote:
> Hi Robert,
>
> this problem persists in the 0.9 release. Using `disableSysoutLogging()`
> works, but I'd rather configure this in the log4j.xml. Is this possible?
>
> Best,
> Stefan
>
> On 14 April 2015 at 20:55, Robert Metzger <rm...@apache.org> wrote:
>
>> You can control the logging behavior from the ExecutionConfig
>> (env.getExecutionConfig()).
>> There is a method (disableSysoutLogging()) that you can use. (In
>> 0.9-SNAPSHOT only).
>>
>> Sorry again that you ran into this issue.
>>
>> On Tue, Apr 14, 2015 at 8:45 PM, Robert Metzger <rm...@apache.org>
>> wrote:
>>
>>> Ah, I see.
>>>
>>> The issue is this line in the JobClient.scala here:
>>> https://github.com/apache/flink/blob/release-0.9.0-milestone-1-rc1/flink-runtime/src/main/scala/org/apache/flink/runtime/client/JobClient.scala#L97
>>>
>>> As you can see, its doing sysout logging.
>>> In the current master, this has been reworked:
>>> https://github.com/apache/flink/blob/69a400fadd258fe0a1ff0b5670a3611fda4c1cdf/flink-runtime/src/main/java/org/apache/flink/runtime/client/JobClientActor.java#L60
>>> (I'm currently looking up how to control the behavior)
>>>
>>> If its possible for you, you can switch to 0.9-SNAPSHOT.
>>>
>>> Best,
>>> Robert
>>>
>>>
>>> On Tue, Apr 14, 2015 at 7:02 PM, Stefan Bunk <stefan.bunk@googlemail.com
>>> > wrote:
>>>
>>>> Hi,
>>>>
>>>> I am also using IntelliJ, and I am starting directly from the IDE.
>>>> Local execution.
>>>> This is what my logging output looks like: [1].
>>>>
>>>> I am getting my logger via:
>>>> val log = org.apache.log4j.Logger.getLogger(getClass)
>>>>
>>>> [1] https://gist.github.com/knub/1c11683601b4eeb5d51b
>>>>
>>>> On 14 April 2015 at 18:47, Robert Metzger <rm...@apache.org> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> how are you starting Flink? Out of the IDE? Using the scripts?
>>>>>
>>>>> I just created a new flink project with the milestone version. Just
>>>>> putting your log4j.xml into the resources folder enabled the logging (I've
>>>>> set it to INFO for flink and it worked).
>>>>> I've used IntelliJ and started the WordCount.java from the quickstart.
>>>>>
>>>>> I think I need more details about your setup to reproduce the issue.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Apr 14, 2015 at 5:52 PM, Stefan Bunk <
>>>>> stefan.bunk@googlemail.com> wrote:
>>>>>
>>>>>> Hi Robert,
>>>>>>
>>>>>> thanks for the info.
>>>>>> Adding the parameter didn't help. My logging file is found and my
>>>>>> logging configuration for my own logging is working (even without the
>>>>>> parameter), it's just that the file in the jar seems to be preferred over
>>>>>> my file.
>>>>>>
>>>>>> Best,
>>>>>> Stefan
>>>>>>
>>>>>> On 14 April 2015 at 17:16, Robert Metzger <rm...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Stefan,
>>>>>>>
>>>>>>> we made a stupid mistake in the 0.9.0-milestone-1 release by
>>>>>>> including our log4j.properties into the flink-runtime jar. its also in the
>>>>>>> fat jar in flink-dist.
>>>>>>>
>>>>>>> Maybe you can pass the name of your log4j file to your application
>>>>>>> with -Dlog4j.configuration=log4j.xml?
>>>>>>>
>>>>>>> The issue is already fixed in the current master but that doesn't
>>>>>>> help 0.9.0-milestone-1 users :(
>>>>>>>
>>>>>>> Best,
>>>>>>> Robert
>>>>>>>
>>>>>>> On Tue, Apr 14, 2015 at 4:49 PM, Stefan Bunk <
>>>>>>> stefan.bunk@googlemail.com> wrote:
>>>>>>>
>>>>>>>> Hi Flinkers,
>>>>>>>>
>>>>>>>> I just switched to 0.9.0-milestone-1, and now I get Flink's logging
>>>>>>>> output again in my console (local execution).
>>>>>>>> I have a log4j.xml under src/main/resources, which says not to log
>>>>>>>> any Flink job progress updates, and which worked fine so far:
>>>>>>>>
>>>>>>>> [...]
>>>>>>>> <logger name="org.apache.flink">
>>>>>>>> <level value="ERROR" />
>>>>>>>> <appender-ref ref="Console" />
>>>>>>>> </logger>
>>>>>>>> [...]
>>>>>>>> See the full file at [1].
>>>>>>>>
>>>>>>>> However, this file seems to be ignored now. What do I need to
>>>>>>>> change?
>>>>>>>>
>>>>>>>> Cheers
>>>>>>>> Stefan
>>>>>>>>
>>>>>>>> [1] https://gist.github.com/knub/bb1b3ffc6b47e2a1376d
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Logging in Flink 0.9.0-milestone-1
Posted by Stefan Bunk <st...@googlemail.com>.
Hi Robert,
this problem persists in the 0.9 release. Using `disableSysoutLogging()`
works, but I'd rather configure this in the log4j.xml. Is this possible?
Best,
Stefan
On 14 April 2015 at 20:55, Robert Metzger <rm...@apache.org> wrote:
> You can control the logging behavior from the ExecutionConfig
> (env.getExecutionConfig()).
> There is a method (disableSysoutLogging()) that you can use. (In
> 0.9-SNAPSHOT only).
>
> Sorry again that you ran into this issue.
>
> On Tue, Apr 14, 2015 at 8:45 PM, Robert Metzger <rm...@apache.org>
> wrote:
>
>> Ah, I see.
>>
>> The issue is this line in the JobClient.scala here:
>> https://github.com/apache/flink/blob/release-0.9.0-milestone-1-rc1/flink-runtime/src/main/scala/org/apache/flink/runtime/client/JobClient.scala#L97
>>
>> As you can see, its doing sysout logging.
>> In the current master, this has been reworked:
>> https://github.com/apache/flink/blob/69a400fadd258fe0a1ff0b5670a3611fda4c1cdf/flink-runtime/src/main/java/org/apache/flink/runtime/client/JobClientActor.java#L60
>> (I'm currently looking up how to control the behavior)
>>
>> If its possible for you, you can switch to 0.9-SNAPSHOT.
>>
>> Best,
>> Robert
>>
>>
>> On Tue, Apr 14, 2015 at 7:02 PM, Stefan Bunk <st...@googlemail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I am also using IntelliJ, and I am starting directly from the IDE. Local
>>> execution.
>>> This is what my logging output looks like: [1].
>>>
>>> I am getting my logger via:
>>> val log = org.apache.log4j.Logger.getLogger(getClass)
>>>
>>> [1] https://gist.github.com/knub/1c11683601b4eeb5d51b
>>>
>>> On 14 April 2015 at 18:47, Robert Metzger <rm...@apache.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> how are you starting Flink? Out of the IDE? Using the scripts?
>>>>
>>>> I just created a new flink project with the milestone version. Just
>>>> putting your log4j.xml into the resources folder enabled the logging (I've
>>>> set it to INFO for flink and it worked).
>>>> I've used IntelliJ and started the WordCount.java from the quickstart.
>>>>
>>>> I think I need more details about your setup to reproduce the issue.
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Apr 14, 2015 at 5:52 PM, Stefan Bunk <
>>>> stefan.bunk@googlemail.com> wrote:
>>>>
>>>>> Hi Robert,
>>>>>
>>>>> thanks for the info.
>>>>> Adding the parameter didn't help. My logging file is found and my
>>>>> logging configuration for my own logging is working (even without the
>>>>> parameter), it's just that the file in the jar seems to be preferred over
>>>>> my file.
>>>>>
>>>>> Best,
>>>>> Stefan
>>>>>
>>>>> On 14 April 2015 at 17:16, Robert Metzger <rm...@apache.org> wrote:
>>>>>
>>>>>> Hi Stefan,
>>>>>>
>>>>>> we made a stupid mistake in the 0.9.0-milestone-1 release by
>>>>>> including our log4j.properties into the flink-runtime jar. its also in the
>>>>>> fat jar in flink-dist.
>>>>>>
>>>>>> Maybe you can pass the name of your log4j file to your application
>>>>>> with -Dlog4j.configuration=log4j.xml?
>>>>>>
>>>>>> The issue is already fixed in the current master but that doesn't
>>>>>> help 0.9.0-milestone-1 users :(
>>>>>>
>>>>>> Best,
>>>>>> Robert
>>>>>>
>>>>>> On Tue, Apr 14, 2015 at 4:49 PM, Stefan Bunk <
>>>>>> stefan.bunk@googlemail.com> wrote:
>>>>>>
>>>>>>> Hi Flinkers,
>>>>>>>
>>>>>>> I just switched to 0.9.0-milestone-1, and now I get Flink's logging
>>>>>>> output again in my console (local execution).
>>>>>>> I have a log4j.xml under src/main/resources, which says not to log
>>>>>>> any Flink job progress updates, and which worked fine so far:
>>>>>>>
>>>>>>> [...]
>>>>>>> <logger name="org.apache.flink">
>>>>>>> <level value="ERROR" />
>>>>>>> <appender-ref ref="Console" />
>>>>>>> </logger>
>>>>>>> [...]
>>>>>>> See the full file at [1].
>>>>>>>
>>>>>>> However, this file seems to be ignored now. What do I need to change?
>>>>>>>
>>>>>>> Cheers
>>>>>>> Stefan
>>>>>>>
>>>>>>> [1] https://gist.github.com/knub/bb1b3ffc6b47e2a1376d
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Logging in Flink 0.9.0-milestone-1
Posted by Robert Metzger <rm...@apache.org>.
You can control the logging behavior from the ExecutionConfig
(env.getExecutionConfig()).
There is a method (disableSysoutLogging()) that you can use. (In
0.9-SNAPSHOT only).
Sorry again that you ran into this issue.
On Tue, Apr 14, 2015 at 8:45 PM, Robert Metzger <rm...@apache.org> wrote:
> Ah, I see.
>
> The issue is this line in the JobClient.scala here:
> https://github.com/apache/flink/blob/release-0.9.0-milestone-1-rc1/flink-runtime/src/main/scala/org/apache/flink/runtime/client/JobClient.scala#L97
>
> As you can see, its doing sysout logging.
> In the current master, this has been reworked:
> https://github.com/apache/flink/blob/69a400fadd258fe0a1ff0b5670a3611fda4c1cdf/flink-runtime/src/main/java/org/apache/flink/runtime/client/JobClientActor.java#L60
> (I'm currently looking up how to control the behavior)
>
> If its possible for you, you can switch to 0.9-SNAPSHOT.
>
> Best,
> Robert
>
>
> On Tue, Apr 14, 2015 at 7:02 PM, Stefan Bunk <st...@googlemail.com>
> wrote:
>
>> Hi,
>>
>> I am also using IntelliJ, and I am starting directly from the IDE. Local
>> execution.
>> This is what my logging output looks like: [1].
>>
>> I am getting my logger via:
>> val log = org.apache.log4j.Logger.getLogger(getClass)
>>
>> [1] https://gist.github.com/knub/1c11683601b4eeb5d51b
>>
>> On 14 April 2015 at 18:47, Robert Metzger <rm...@apache.org> wrote:
>>
>>> Hi,
>>>
>>> how are you starting Flink? Out of the IDE? Using the scripts?
>>>
>>> I just created a new flink project with the milestone version. Just
>>> putting your log4j.xml into the resources folder enabled the logging (I've
>>> set it to INFO for flink and it worked).
>>> I've used IntelliJ and started the WordCount.java from the quickstart.
>>>
>>> I think I need more details about your setup to reproduce the issue.
>>>
>>>
>>>
>>>
>>> On Tue, Apr 14, 2015 at 5:52 PM, Stefan Bunk <stefan.bunk@googlemail.com
>>> > wrote:
>>>
>>>> Hi Robert,
>>>>
>>>> thanks for the info.
>>>> Adding the parameter didn't help. My logging file is found and my
>>>> logging configuration for my own logging is working (even without the
>>>> parameter), it's just that the file in the jar seems to be preferred over
>>>> my file.
>>>>
>>>> Best,
>>>> Stefan
>>>>
>>>> On 14 April 2015 at 17:16, Robert Metzger <rm...@apache.org> wrote:
>>>>
>>>>> Hi Stefan,
>>>>>
>>>>> we made a stupid mistake in the 0.9.0-milestone-1 release by including
>>>>> our log4j.properties into the flink-runtime jar. its also in the fat jar in
>>>>> flink-dist.
>>>>>
>>>>> Maybe you can pass the name of your log4j file to your application
>>>>> with -Dlog4j.configuration=log4j.xml?
>>>>>
>>>>> The issue is already fixed in the current master but that doesn't help
>>>>> 0.9.0-milestone-1 users :(
>>>>>
>>>>> Best,
>>>>> Robert
>>>>>
>>>>> On Tue, Apr 14, 2015 at 4:49 PM, Stefan Bunk <
>>>>> stefan.bunk@googlemail.com> wrote:
>>>>>
>>>>>> Hi Flinkers,
>>>>>>
>>>>>> I just switched to 0.9.0-milestone-1, and now I get Flink's logging
>>>>>> output again in my console (local execution).
>>>>>> I have a log4j.xml under src/main/resources, which says not to log
>>>>>> any Flink job progress updates, and which worked fine so far:
>>>>>>
>>>>>> [...]
>>>>>> <logger name="org.apache.flink">
>>>>>> <level value="ERROR" />
>>>>>> <appender-ref ref="Console" />
>>>>>> </logger>
>>>>>> [...]
>>>>>> See the full file at [1].
>>>>>>
>>>>>> However, this file seems to be ignored now. What do I need to change?
>>>>>>
>>>>>> Cheers
>>>>>> Stefan
>>>>>>
>>>>>> [1] https://gist.github.com/knub/bb1b3ffc6b47e2a1376d
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Logging in Flink 0.9.0-milestone-1
Posted by Robert Metzger <rm...@apache.org>.
Ah, I see.
The issue is this line in the JobClient.scala here:
https://github.com/apache/flink/blob/release-0.9.0-milestone-1-rc1/flink-runtime/src/main/scala/org/apache/flink/runtime/client/JobClient.scala#L97
As you can see, its doing sysout logging.
In the current master, this has been reworked:
https://github.com/apache/flink/blob/69a400fadd258fe0a1ff0b5670a3611fda4c1cdf/flink-runtime/src/main/java/org/apache/flink/runtime/client/JobClientActor.java#L60
(I'm currently looking up how to control the behavior)
If its possible for you, you can switch to 0.9-SNAPSHOT.
Best,
Robert
On Tue, Apr 14, 2015 at 7:02 PM, Stefan Bunk <st...@googlemail.com>
wrote:
> Hi,
>
> I am also using IntelliJ, and I am starting directly from the IDE. Local
> execution.
> This is what my logging output looks like: [1].
>
> I am getting my logger via:
> val log = org.apache.log4j.Logger.getLogger(getClass)
>
> [1] https://gist.github.com/knub/1c11683601b4eeb5d51b
>
> On 14 April 2015 at 18:47, Robert Metzger <rm...@apache.org> wrote:
>
>> Hi,
>>
>> how are you starting Flink? Out of the IDE? Using the scripts?
>>
>> I just created a new flink project with the milestone version. Just
>> putting your log4j.xml into the resources folder enabled the logging (I've
>> set it to INFO for flink and it worked).
>> I've used IntelliJ and started the WordCount.java from the quickstart.
>>
>> I think I need more details about your setup to reproduce the issue.
>>
>>
>>
>>
>> On Tue, Apr 14, 2015 at 5:52 PM, Stefan Bunk <st...@googlemail.com>
>> wrote:
>>
>>> Hi Robert,
>>>
>>> thanks for the info.
>>> Adding the parameter didn't help. My logging file is found and my
>>> logging configuration for my own logging is working (even without the
>>> parameter), it's just that the file in the jar seems to be preferred over
>>> my file.
>>>
>>> Best,
>>> Stefan
>>>
>>> On 14 April 2015 at 17:16, Robert Metzger <rm...@apache.org> wrote:
>>>
>>>> Hi Stefan,
>>>>
>>>> we made a stupid mistake in the 0.9.0-milestone-1 release by including
>>>> our log4j.properties into the flink-runtime jar. its also in the fat jar in
>>>> flink-dist.
>>>>
>>>> Maybe you can pass the name of your log4j file to your application with
>>>> -Dlog4j.configuration=log4j.xml?
>>>>
>>>> The issue is already fixed in the current master but that doesn't help
>>>> 0.9.0-milestone-1 users :(
>>>>
>>>> Best,
>>>> Robert
>>>>
>>>> On Tue, Apr 14, 2015 at 4:49 PM, Stefan Bunk <
>>>> stefan.bunk@googlemail.com> wrote:
>>>>
>>>>> Hi Flinkers,
>>>>>
>>>>> I just switched to 0.9.0-milestone-1, and now I get Flink's logging
>>>>> output again in my console (local execution).
>>>>> I have a log4j.xml under src/main/resources, which says not to log any
>>>>> Flink job progress updates, and which worked fine so far:
>>>>>
>>>>> [...]
>>>>> <logger name="org.apache.flink">
>>>>> <level value="ERROR" />
>>>>> <appender-ref ref="Console" />
>>>>> </logger>
>>>>> [...]
>>>>> See the full file at [1].
>>>>>
>>>>> However, this file seems to be ignored now. What do I need to change?
>>>>>
>>>>> Cheers
>>>>> Stefan
>>>>>
>>>>> [1] https://gist.github.com/knub/bb1b3ffc6b47e2a1376d
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Logging in Flink 0.9.0-milestone-1
Posted by Stefan Bunk <st...@googlemail.com>.
Hi,
I am also using IntelliJ, and I am starting directly from the IDE. Local
execution.
This is what my logging output looks like: [1].
I am getting my logger via:
val log = org.apache.log4j.Logger.getLogger(getClass)
[1] https://gist.github.com/knub/1c11683601b4eeb5d51b
On 14 April 2015 at 18:47, Robert Metzger <rm...@apache.org> wrote:
> Hi,
>
> how are you starting Flink? Out of the IDE? Using the scripts?
>
> I just created a new flink project with the milestone version. Just
> putting your log4j.xml into the resources folder enabled the logging (I've
> set it to INFO for flink and it worked).
> I've used IntelliJ and started the WordCount.java from the quickstart.
>
> I think I need more details about your setup to reproduce the issue.
>
>
>
>
> On Tue, Apr 14, 2015 at 5:52 PM, Stefan Bunk <st...@googlemail.com>
> wrote:
>
>> Hi Robert,
>>
>> thanks for the info.
>> Adding the parameter didn't help. My logging file is found and my logging
>> configuration for my own logging is working (even without the parameter),
>> it's just that the file in the jar seems to be preferred over my file.
>>
>> Best,
>> Stefan
>>
>> On 14 April 2015 at 17:16, Robert Metzger <rm...@apache.org> wrote:
>>
>>> Hi Stefan,
>>>
>>> we made a stupid mistake in the 0.9.0-milestone-1 release by including
>>> our log4j.properties into the flink-runtime jar. its also in the fat jar in
>>> flink-dist.
>>>
>>> Maybe you can pass the name of your log4j file to your application with
>>> -Dlog4j.configuration=log4j.xml?
>>>
>>> The issue is already fixed in the current master but that doesn't help
>>> 0.9.0-milestone-1 users :(
>>>
>>> Best,
>>> Robert
>>>
>>> On Tue, Apr 14, 2015 at 4:49 PM, Stefan Bunk <stefan.bunk@googlemail.com
>>> > wrote:
>>>
>>>> Hi Flinkers,
>>>>
>>>> I just switched to 0.9.0-milestone-1, and now I get Flink's logging
>>>> output again in my console (local execution).
>>>> I have a log4j.xml under src/main/resources, which says not to log any
>>>> Flink job progress updates, and which worked fine so far:
>>>>
>>>> [...]
>>>> <logger name="org.apache.flink">
>>>> <level value="ERROR" />
>>>> <appender-ref ref="Console" />
>>>> </logger>
>>>> [...]
>>>> See the full file at [1].
>>>>
>>>> However, this file seems to be ignored now. What do I need to change?
>>>>
>>>> Cheers
>>>> Stefan
>>>>
>>>> [1] https://gist.github.com/knub/bb1b3ffc6b47e2a1376d
>>>>
>>>
>>>
>>
>
Re: Logging in Flink 0.9.0-milestone-1
Posted by Robert Metzger <rm...@apache.org>.
Hi,
how are you starting Flink? Out of the IDE? Using the scripts?
I just created a new flink project with the milestone version. Just putting
your log4j.xml into the resources folder enabled the logging (I've set it
to INFO for flink and it worked).
I've used IntelliJ and started the WordCount.java from the quickstart.
I think I need more details about your setup to reproduce the issue.
On Tue, Apr 14, 2015 at 5:52 PM, Stefan Bunk <st...@googlemail.com>
wrote:
> Hi Robert,
>
> thanks for the info.
> Adding the parameter didn't help. My logging file is found and my logging
> configuration for my own logging is working (even without the parameter),
> it's just that the file in the jar seems to be preferred over my file.
>
> Best,
> Stefan
>
> On 14 April 2015 at 17:16, Robert Metzger <rm...@apache.org> wrote:
>
>> Hi Stefan,
>>
>> we made a stupid mistake in the 0.9.0-milestone-1 release by including
>> our log4j.properties into the flink-runtime jar. its also in the fat jar in
>> flink-dist.
>>
>> Maybe you can pass the name of your log4j file to your application with -
>> Dlog4j.configuration=log4j.xml?
>>
>> The issue is already fixed in the current master but that doesn't help
>> 0.9.0-milestone-1 users :(
>>
>> Best,
>> Robert
>>
>> On Tue, Apr 14, 2015 at 4:49 PM, Stefan Bunk <st...@googlemail.com>
>> wrote:
>>
>>> Hi Flinkers,
>>>
>>> I just switched to 0.9.0-milestone-1, and now I get Flink's logging
>>> output again in my console (local execution).
>>> I have a log4j.xml under src/main/resources, which says not to log any
>>> Flink job progress updates, and which worked fine so far:
>>>
>>> [...]
>>> <logger name="org.apache.flink">
>>> <level value="ERROR" />
>>> <appender-ref ref="Console" />
>>> </logger>
>>> [...]
>>> See the full file at [1].
>>>
>>> However, this file seems to be ignored now. What do I need to change?
>>>
>>> Cheers
>>> Stefan
>>>
>>> [1] https://gist.github.com/knub/bb1b3ffc6b47e2a1376d
>>>
>>
>>
>
Re: Logging in Flink 0.9.0-milestone-1
Posted by Stefan Bunk <st...@googlemail.com>.
Hi Robert,
thanks for the info.
Adding the parameter didn't help. My logging file is found and my logging
configuration for my own logging is working (even without the parameter),
it's just that the file in the jar seems to be preferred over my file.
Best,
Stefan
On 14 April 2015 at 17:16, Robert Metzger <rm...@apache.org> wrote:
> Hi Stefan,
>
> we made a stupid mistake in the 0.9.0-milestone-1 release by including our
> log4j.properties into the flink-runtime jar. its also in the fat jar in
> flink-dist.
>
> Maybe you can pass the name of your log4j file to your application with -
> Dlog4j.configuration=log4j.xml?
>
> The issue is already fixed in the current master but that doesn't help
> 0.9.0-milestone-1 users :(
>
> Best,
> Robert
>
> On Tue, Apr 14, 2015 at 4:49 PM, Stefan Bunk <st...@googlemail.com>
> wrote:
>
>> Hi Flinkers,
>>
>> I just switched to 0.9.0-milestone-1, and now I get Flink's logging
>> output again in my console (local execution).
>> I have a log4j.xml under src/main/resources, which says not to log any
>> Flink job progress updates, and which worked fine so far:
>>
>> [...]
>> <logger name="org.apache.flink">
>> <level value="ERROR" />
>> <appender-ref ref="Console" />
>> </logger>
>> [...]
>> See the full file at [1].
>>
>> However, this file seems to be ignored now. What do I need to change?
>>
>> Cheers
>> Stefan
>>
>> [1] https://gist.github.com/knub/bb1b3ffc6b47e2a1376d
>>
>
>
Re: Logging in Flink 0.9.0-milestone-1
Posted by Robert Metzger <rm...@apache.org>.
Hi Stefan,
we made a stupid mistake in the 0.9.0-milestone-1 release by including our
log4j.properties into the flink-runtime jar. its also in the fat jar in
flink-dist.
Maybe you can pass the name of your log4j file to your application with -
Dlog4j.configuration=log4j.xml?
The issue is already fixed in the current master but that doesn't help
0.9.0-milestone-1 users :(
Best,
Robert
On Tue, Apr 14, 2015 at 4:49 PM, Stefan Bunk <st...@googlemail.com>
wrote:
> Hi Flinkers,
>
> I just switched to 0.9.0-milestone-1, and now I get Flink's logging output
> again in my console (local execution).
> I have a log4j.xml under src/main/resources, which says not to log any
> Flink job progress updates, and which worked fine so far:
>
> [...]
> <logger name="org.apache.flink">
> <level value="ERROR" />
> <appender-ref ref="Console" />
> </logger>
> [...]
> See the full file at [1].
>
> However, this file seems to be ignored now. What do I need to change?
>
> Cheers
> Stefan
>
> [1] https://gist.github.com/knub/bb1b3ffc6b47e2a1376d
>