You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/01/06 09:36:31 UTC
[GitHub] [incubator-dolphinscheduler] wuzhjian opened a new issue #4397: [Question] Question title
wuzhjian opened a new issue #4397:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4397
设置flink节点，处理实时数据的时候，如何设置checkpoint，当任务彻底挂掉需要人为干预启动的时候，通过指定的checkpoint目录重启。
当任务需要修改部分代码重启，在将任务下线的时候指定savepoint，flink on yarn 目前停止任务是直接通过yarn application -kill，并不能指定savepoint
**Which version of DolphinScheduler:**
-[1.3.3-preview]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen commented on issue #4397: [Question] 如何在设置flink 节点，任务重启的时候指定checkpoint目录
Posted by GitBox <gi...@apache.org>.
chengshiwen commented on issue #4397:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4397#issuecomment-756507417
@wuzhjian
Under `yarn cluster` deploy mode, you can
- set `-yD state.backend=filesystem -yD state.checkpoints.dir=hdfs://namenode:9000/flink/checkpoints -yD state.savepoints.dir=hdfs://namenode:9000/flink/savepoints` in `other parameters` (其它参数) to configure state options
- trigger a savepoint manually by `flink savepoint <jobId> [savepointDirectory] -yid <yarnAppId>`
- set `-s <savepointPath> -n` in `other parameters` to restore from the specific savepoint path
In other deploy mode such as `local` and `session`, you can only set `state.backend`, `state.checkpoints.dir` and `state.savepoints.dir` in the configuration file `flink-conf.yaml`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] xingchun-chen commented on issue #4397: [Question] 如何在设置flink 节点，任务重启的时候指定checkpoint目录
Posted by GitBox <gi...@apache.org>.
xingchun-chen commented on issue #4397:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4397#issuecomment-756015471
English is global and necessary, please update the title in English, thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen commented on issue #4397: [Question] How to specify the checkpoint directory when setting up flink nodes and task restart
Posted by GitBox <gi...@apache.org>.
chengshiwen commented on issue #4397:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4397#issuecomment-817107845
Close this question since it has been replied
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen closed issue #4397: [Question] How to specify the checkpoint directory when setting up flink nodes and task restart
Posted by GitBox <gi...@apache.org>.
chengshiwen closed issue #4397:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4397
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org