You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Xintong Song (Jira)" <ji...@apache.org> on 2022/11/24 02:05:00 UTC
[jira] [Updated] (FLINK-29530) CommittableMessageTypeInfo#toString may force client to have S3 credentials
[ https://issues.apache.org/jira/browse/FLINK-29530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xintong Song updated FLINK-29530:
---------------------------------
Fix Version/s: 1.15.4
(was: 1.15.3)
> CommittableMessageTypeInfo#toString may force client to have S3 credentials
> ---------------------------------------------------------------------------
>
> Key: FLINK-29530
> URL: https://issues.apache.org/jira/browse/FLINK-29530
> Project: Flink
> Issue Type: Technical Debt
> Components: Connectors / FileSystem
> Affects Versions: 1.15.0
> Reporter: Chesnay Schepler
> Priority: Critical
> Fix For: 1.17.0, 1.16.1, 1.15.4
>
>
> The {{toString}} implementation calls the committable serializer factory, which in the case of the {{FileSink}} loads a filesystem and thus may require credentials.
> As a result something as basic as logging the type info can force the client to require S3 credentials.
> It shouldn't have such side-effects......
> Stacktrace that a user provided for another issue:
> {code}
> Exception in thread "main" org.apache.flink.util.FlinkRuntimeException: Could not create committable serializer.
> at org.apache.flink.connector.file.sink.FileSink.getCommittableSerializer(FileSink.java:180)
> at org.apache.flink.streaming.api.connector.sink2.CommittableMessageTypeInfo.toString(CommittableMessageTypeInfo.java:120)
> at java.base/java.lang.String.valueOf(String.java:2951)
> at java.base/java.lang.StringBuilder.append(StringBuilder.java:172)
> at org.apache.flink.api.dag.Transformation.toString(Transformation.java:556)
> at java.base/java.lang.String.valueOf(String.java:2951)
> at java.base/java.lang.StringBuilder.append(StringBuilder.java:172)
> at org.apache.flink.streaming.api.graph.StreamGraphGenerator.transform(StreamGraphGenerator.java:506)
> at org.apache.flink.streaming.api.graph.StreamGraphGenerator.access$200(StreamGraphGenerator.java:131)
> at org.apache.flink.streaming.api.graph.StreamGraphGenerator$ContextImpl.transform(StreamGraphGenerator.java:933)
> at java.base/java.lang.Iterable.forEach(Iterable.java:75)
> at org.apache.flink.streaming.runtime.translators.SinkTransformationTranslator$SinkExpander.expand(SinkTransformationTranslator.java:157)
> at org.apache.flink.streaming.runtime.translators.SinkTransformationTranslator$SinkExpander.access$000(SinkTransformationTranslator.java:99)
> at org.apache.flink.streaming.runtime.translators.SinkTransformationTranslator.translateInternal(SinkTransformationTranslator.java:89)
> at org.apache.flink.streaming.runtime.translators.SinkTransformationTranslator.translateForStreaming(SinkTransformationTranslator.java:77)
> at org.apache.flink.streaming.runtime.translators.SinkTransformationTranslator.translateForStreaming(SinkTransformationTranslator.java:61)
> at org.apache.flink.streaming.api.graph.StreamGraphGenerator.translate(StreamGraphGenerator.java:825)
> at org.apache.flink.streaming.api.graph.StreamGraphGenerator.transform(StreamGraphGenerator.java:555)
> at org.apache.flink.streaming.api.graph.StreamGraphGenerator.generate(StreamGraphGenerator.java:316)
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getStreamGraph(StreamExecutionEnvironment.java:2135)
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getStreamGraph(StreamExecutionEnvironment.java:2121)
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1967)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)