You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Yangze Guo (Jira)" <ji...@apache.org> on 2020/05/08 12:45:00 UTC
[jira] [Updated] (FLINK-16548) Expose consistent environment
variable to identify the component name and resource id of jm/tm
[ https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yangze Guo updated FLINK-16548:
-------------------------------
Summary: Expose consistent environment variable to identify the component name and resource id of jm/tm (was: [Requirement] Consistent id to identify jm/tm in different cluster modes)
> Expose consistent environment variable to identify the component name and resource id of jm/tm
> ----------------------------------------------------------------------------------------------
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Configuration
> Reporter: hejianchao
> Priority: Major
>
> Imagine that there is now s3-log-appender.jar, and logs can be written to s3 with proper configuration.
> Then, in the same flink job, the exact same log4j configuration is used. How to make the key written to s3 for different jm / tm carry the id that uniquely identifies jm / tm? such as:
> s3://bucket/flink-logs/jobId/${jm-id}/logfile
> s3://bucket/flink-logs/jobId/${tm-1-id}/logfile
> s3://bucket/flink-logs/jobId/${tm-2-id}/logfile
> For the s3-log-appender here, it needs a way to sense the value of ${jm-id}/${tm-1-id}/${tm-2-id}, and it has the following claims :
> 1. This value can be used to locate the actual running jm, tm
> 2. The value is independent of cluster mode, whether standalone, yarn or kubernetes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)