You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Reo Lei <le...@gmail.com> on 2020/03/20 10:55:22 UTC
Some question about flink temp files
Hi all,
Recently, I found flink tmp files(localState,
blobStore-*, flink-dist-cach-*, flink-io-*, flink-netty-shuffle-* etc.) has
been grown to a total of about 6GB size. I have no idea what usage abot
this files, and How big it will grow.
So, I have some questions about the tmep files of flink as follow:
1, what is the usage of this tmp files?
2, Is there have any mechanism of flink to manage this files?
3, Have any situation I need to delete the useless files manually？
4, I know I can specify the temp files path by "io.tmp.dirs" config for
flink cluster. But when I use LocalEnvironment, how can I config this path
instead of use the default path("/tmp")?
Thanks.
BR,
Reo
Re: Some question about flink temp files
Posted by Reo Lei <le...@gmail.com>.
Thank you Roman!
That is very helpful! Thank you!
BR,
Reo
Khachatryan Roman <kh...@gmail.com> 于2020年3月20日周五 下午11:13写道：
> Hi Reo,
>
> Please find the answers to your questions below.
>
> > 1, what is the usage of this tmp files?
> These files are used by Flink internally for things like caching state
> locally, storing jars and so on. They are not intended for the end-user.
> > 2, Is there have any mechanism of flink to manage this files?
> Yes, they are created and deleted by Flink when not used.
> > 3, Have any situation I need to delete the useless files manually？
> No.
> > 4, I know I can specify the temp files path by "io.tmp.dirs" config for
> flink cluster. But when I use LocalEnvironment, how can I config this path
> instead of use the default path("/tmp")?
> Please use "java.io.tmpdir" *system property* for standalone deployment
> [1].
>
> [1]
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/config.html
>
> Regards,
> Roman
>
>
> On Fri, Mar 20, 2020 at 11:55 AM Reo Lei <le...@gmail.com> wrote:
>
>> Hi all,
>>
>> Recently, I found flink tmp files(localState,
>> blobStore-*, flink-dist-cach-*, flink-io-*, flink-netty-shuffle-* etc.) has
>> been grown to a total of about 6GB size. I have no idea what usage abot
>> this files, and How big it will grow.
>>
>> So, I have some questions about the tmep files of flink as follow:
>>
>> 1, what is the usage of this tmp files?
>> 2, Is there have any mechanism of flink to manage this files?
>> 3, Have any situation I need to delete the useless files manually？
>> 4, I know I can specify the temp files path by "io.tmp.dirs" config for
>> flink cluster. But when I use LocalEnvironment, how can I config this
>> path instead of use the default path("/tmp")?
>>
>> Thanks.
>>
>> BR,
>> Reo
>>
>
Re: Some question about flink temp files
Posted by Khachatryan Roman <kh...@gmail.com>.
Hi Reo,
Please find the answers to your questions below.
> 1, what is the usage of this tmp files?
These files are used by Flink internally for things like caching state
locally, storing jars and so on. They are not intended for the end-user.
> 2, Is there have any mechanism of flink to manage this files?
Yes, they are created and deleted by Flink when not used.
> 3, Have any situation I need to delete the useless files manually？
No.
> 4, I know I can specify the temp files path by "io.tmp.dirs" config for
flink cluster. But when I use LocalEnvironment, how can I config this path
instead of use the default path("/tmp")?
Please use "java.io.tmpdir" *system property* for standalone deployment [1].
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/config.html
Regards,
Roman
On Fri, Mar 20, 2020 at 11:55 AM Reo Lei <le...@gmail.com> wrote:
> Hi all,
>
> Recently, I found flink tmp files(localState,
> blobStore-*, flink-dist-cach-*, flink-io-*, flink-netty-shuffle-* etc.) has
> been grown to a total of about 6GB size. I have no idea what usage abot
> this files, and How big it will grow.
>
> So, I have some questions about the tmep files of flink as follow:
>
> 1, what is the usage of this tmp files?
> 2, Is there have any mechanism of flink to manage this files?
> 3, Have any situation I need to delete the useless files manually？
> 4, I know I can specify the temp files path by "io.tmp.dirs" config for
> flink cluster. But when I use LocalEnvironment, how can I config this
> path instead of use the default path("/tmp")?
>
> Thanks.
>
> BR,
> Reo
>