You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Vinay Devadiga (Jira)" <ji...@apache.org> on 2022/06/09 16:27:00 UTC
[jira] [Comment Edited] (FLINK-27967) Unable to inject entropy on s3 prefix when doing a save point in Flink
[ https://issues.apache.org/jira/browse/FLINK-27967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17552308#comment-17552308 ]
Vinay Devadiga edited comment on FLINK-27967 at 6/9/22 4:26 PM:
----------------------------------------------------------------
[~martijnvisser] Savepoint completed. Path: s3://vinaydevuswest2/{_}entropy{_}/flink/savepoint-data/savepoint-238579-bc904c407d82 this is the path where it got saved according to the documentation - f entropy injection is activated, a configured substring in the path is replaced with random characters. For example, path {{s3://my-bucket/checkpoints/{_}entropy{_}/dashboard-job/}} would be replaced by something like {{s3://my-bucket/checkpoints/gf36ikvg/dashboard-job/}}
{{}}
was (Author: JIRAUSER290527):
Savepoint completed. Path: s3://vinaydevuswest2/_entropy_/flink/savepoint-data/savepoint-238579-bc904c407d82 this is the path where it got saved according to the documentation - f entropy injection is activated, a configured substring in the path is replaced with random characters. For example, path {{s3://my-bucket/checkpoints/_entropy_/dashboard-job/}} would be replaced by something like {{s3://my-bucket/checkpoints/gf36ikvg/dashboard-job/}}
{{}}
> Unable to inject entropy on s3 prefix when doing a save point in Flink
> ----------------------------------------------------------------------
>
> Key: FLINK-27967
> URL: https://issues.apache.org/jira/browse/FLINK-27967
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
> Affects Versions: 1.14.0
> Reporter: Vinay Devadiga
> Priority: Major
> Attachments: Screenshot 2022-06-09 at 12.10.51 PM.png
>
>
> Hi while using flink 1.14.0,I ran the example job /examples/streaming/StateMachineExample.jar and the job submitted successfully and then I tried to save point it to s3 with entropy enabled but the entropy key was not respected here are my configurations.Can anyone please guide me through issue, I an trying to go through the code but could not find anything sustainable.
> |flink-conf|fs.allowed-fallback-filesystems|s3|Cluster configuration| |
> |flink-conf|execution.checkpointing.unaligned|true|Cluster configuration| |
> |flink-conf|state.backend.incremental|true|Cluster configuration| |
> |flink-conf|execution.checkpointing.timeout|600min|Cluster configuration| |
> |flink-conf|execution.checkpointing.externalized-checkpoint-retention|RETAIN_ON_CANCELLATION|Cluster configuration| |
> |flink-conf|state.backend|rocksdb|Cluster configuration| |
> |flink-conf|s3.entropy.key|_entropy_|Cluster configuration| |
> |flink-conf|state.checkpoints.dir|s3://vinaydevuswest2/_entropy_/flink/checkpoint-data/|Cluster configuration| |
> |flink-conf|execution.checkpointing.max-concurrent-checkpoints|1|Cluster configuration| |
> |flink-conf|execution.checkpointing.min-pause|5000|Cluster configuration| |
> |flink-conf|execution.checkpointing.checkpoints-after-tasks-finish.enabled|true|Cluster configuration| |
> |flink-conf|state.savepoints.dir|s3://vinaydevuswest2/_entropy_/flink/savepoint-data/|Cluster configuration| |
> |flink-conf|state.storage.fs.memory-threshold|0|Cluster configuration| |
> |flink-conf|s3.entropy.length|4|Cluster configuration| |
> |flink-conf|execution.checkpointing.tolerable-failed-checkpoints|30|Cluster configuration| |
> |flink-conf|execution.checkpointing.mode|EXACTLY_ONCE|Cluster configuration|
--
This message was sent by Atlassian Jira
(v8.20.7#820007)