You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Alexander Smirnov <al...@gmail.com> on 2018/08/01 15:19:28 UTC
Flink log and out files
Hi,
could you please explain the difference between *.log and *.out files in
Flink?
What information is supposed to be in each of them?
Is "log" a subset of "out"?
How do I setup rotation with gzipping?
Thank you,
Alex
Re: Flink log and out files
Posted by vino yang <ya...@gmail.com>.
Hi Alexander,
Yes, usually if you configure the log output as a .log file, the .out file
will not have content, but sometimes some fatal exceptions, such as JVM
exit exceptions, may be printed to the .out file.
If Flink logs appear in your .out file, and it seems that the format of the
log is inconsistent with the format you configured in the configuration
file, it is likely that SLF4j detects that there are multiple interface
implementation bindings in your jar dependency.
This will cause the log output to be abnormal, which we have encountered
before.
Thanks, vino.
2018-08-02 3:27 GMT+08:00 Alexander Smirnov <al...@gmail.com>:
> thanks guys,
>
> So, is it a correct statement - if my job doesn't write anything to
> stdout, the "*.out" file should be empty?
>
> for some reason it contains the same info as "log" and much more.
>
> For the "log" files, I can control rotation via log4j configuration, but
> how do I setup rotation for "out" files?
> Or, how do I disable them at all?
>
> I'm using 1.4.2
>
> Thank you,
> Alex
>
> On Wed, Aug 1, 2018 at 7:00 PM Andrey Zagrebin <an...@data-artisans.com>
> wrote:
>
>> Hi Alexander,
>>
>> there is also a doc link where log configuration is described:
>> https://ci.apache.org/projects/flink/flink-docs-release-1.5/monitoring/
>> logging.html
>> You can modify log configuration in conf directory according to logging
>> framework docs.
>>
>> Cheers,
>> Andrey
>>
>>
>> On 1 Aug 2018, at 17:30, vino yang <ya...@gmail.com> wrote:
>>
>> Hi Alexander:
>>
>> .log and .out are different. Usually, the .log file stores the log
>> information output by the log framework. Flink uses slf4j as the log
>> interface and supports log4j and logback configurations. The .out file
>> stores the STDOUT information. This information is usually output by you
>> calling some APIs such as the print sink API.
>>
>> Thanks, vino.
>>
>> 2018-08-01 23:19 GMT+08:00 Alexander Smirnov <
>> alexander.smirnoff@gmail.com>:
>>
>>> Hi,
>>>
>>> could you please explain the difference between *.log and *.out files in
>>> Flink?
>>> What information is supposed to be in each of them?
>>> Is "log" a subset of "out"?
>>> How do I setup rotation with gzipping?
>>>
>>> Thank you,
>>> Alex
>>>
>>
>>
>>
Re: Flink log and out files
Posted by Alexander Smirnov <al...@gmail.com>.
thanks guys,
So, is it a correct statement - if my job doesn't write anything to stdout,
the "*.out" file should be empty?
for some reason it contains the same info as "log" and much more.
For the "log" files, I can control rotation via log4j configuration, but
how do I setup rotation for "out" files?
Or, how do I disable them at all?
I'm using 1.4.2
Thank you,
Alex
On Wed, Aug 1, 2018 at 7:00 PM Andrey Zagrebin <an...@data-artisans.com>
wrote:
> Hi Alexander,
>
> there is also a doc link where log configuration is described:
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.5/monitoring/logging.html
> You can modify log configuration in conf directory according to logging
> framework docs.
>
> Cheers,
> Andrey
>
>
> On 1 Aug 2018, at 17:30, vino yang <ya...@gmail.com> wrote:
>
> Hi Alexander:
>
> .log and .out are different. Usually, the .log file stores the log
> information output by the log framework. Flink uses slf4j as the log
> interface and supports log4j and logback configurations. The .out file
> stores the STDOUT information. This information is usually output by you
> calling some APIs such as the print sink API.
>
> Thanks, vino.
>
> 2018-08-01 23:19 GMT+08:00 Alexander Smirnov <alexander.smirnoff@gmail.com
> >:
>
>> Hi,
>>
>> could you please explain the difference between *.log and *.out files in
>> Flink?
>> What information is supposed to be in each of them?
>> Is "log" a subset of "out"?
>> How do I setup rotation with gzipping?
>>
>> Thank you,
>> Alex
>>
>
>
>
Re: Flink log and out files
Posted by Andrey Zagrebin <an...@data-artisans.com>.
Hi Alexander,
there is also a doc link where log configuration is described:
https://ci.apache.org/projects/flink/flink-docs-release-1.5/monitoring/logging.html <https://ci.apache.org/projects/flink/flink-docs-release-1.5/monitoring/logging.html>
You can modify log configuration in conf directory according to logging framework docs.
Cheers,
Andrey
> On 1 Aug 2018, at 17:30, vino yang <ya...@gmail.com> wrote:
>
> Hi Alexander:
>
> .log and .out are different. Usually, the .log file stores the log information output by the log framework. Flink uses slf4j as the log interface and supports log4j and logback configurations. The .out file stores the STDOUT information. This information is usually output by you calling some APIs such as the print sink API.
>
> Thanks, vino.
>
> 2018-08-01 23:19 GMT+08:00 Alexander Smirnov <alexander.smirnoff@gmail.com <ma...@gmail.com>>:
> Hi,
>
> could you please explain the difference between *.log and *.out files in Flink?
> What information is supposed to be in each of them?
> Is "log" a subset of "out"?
> How do I setup rotation with gzipping?
>
> Thank you,
> Alex
>
Re: Flink log and out files
Posted by vino yang <ya...@gmail.com>.
Hi Alexander:
.log and .out are different. Usually, the .log file stores the log
information output by the log framework. Flink uses slf4j as the log
interface and supports log4j and logback configurations. The .out file
stores the STDOUT information. This information is usually output by you
calling some APIs such as the print sink API.
Thanks, vino.
2018-08-01 23:19 GMT+08:00 Alexander Smirnov <al...@gmail.com>:
> Hi,
>
> could you please explain the difference between *.log and *.out files in
> Flink?
> What information is supposed to be in each of them?
> Is "log" a subset of "out"?
> How do I setup rotation with gzipping?
>
> Thank you,
> Alex
>