[jira] [Updated] (FLINK-37069) Cross-team verification for "Disaggregated State Management"

["Zakelly Lan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-37069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zakelly Lan updated FLINK-37069:
--------------------------------
    Description: 
Instructions:

First of all, please read the related documents briefly (still under review, will replace with formal links if merged):
* Disaggregated State Management: https://github.com/apache/flink/pull/26107/files#diff-bfa19e04bb5c3487c3e9bf514d61c0fa8bb973950fb0ad0e3d4a6898a99b83e3
* State V2: https://github.com/apache/flink/pull/26107/files#diff-5d1147987fecbda329132403c1d92384575be220092995c4be491e12b8c50cc9
* ForSt State Backend: https://github.com/apache/flink/pull/26107/files#diff-b7c52c06f6ed4d5af6f230d11ba23ea051bf4a08c589d98392143f080c468a87

For the SQL part, verification goes in FLINK-37068, we mainly focus on Datastream jobs and APIs here.

1. Make sure you are verifying this on master branch, since we have fixed several bugs since the rc0 package.
2. Choose one example in `flink-examples-streaming`. Most of the jobs has been rewritten using new API. Here we take `StateMachineExample` as an example.
3. Compile and run `StateMachineExample` in proper environment (I suggest a standalone session cluster or yarn), make sure you have the following command line params:
{code:bash}
./flink run xxxxxxxxx \
  --backend forst \
  --checkpoint-dir s3://your/cp/dir \
  --incremental-checkpoints true
{code}
Or set via `config.yaml`.
{code:yaml}
state.backend.type: forst
execution.checkpointing.incremental: true
execution.checkpointing.dir: s3://your-bucket/flink-checkpoints
{code}

4. Check the job is running smoothly, the periodic checkpoints are successfully taken.
5. Stop the job and restart from the latest checkpoint.

It would be great if you could write your own job using State V2 API, and follow the above Step 3~5. It is important to check whether there is any bug in new State APIs.


  was:
This is for cross-team verification of the release 2.0 work item: "Disaggregated State Management"

Please complete the following steps before Jan 31.
 # Replace the description of the ticket with instructions for the testers.
 ** If the work item does not need cross-team verification, please comment that explicitly and close the ticket.
 # Un-assign yourself after adding the instructions.
 # Either find a cross-team tester who is willing to help and assign to the ticket, or reach out to the release manager [~xtsong].

Thanks for your contributions. For any questions, feel free to reach out to the release manager [~xtsong].


> Cross-team verification for "Disaggregated State Management"
> ------------------------------------------------------------
>
>                 Key: FLINK-37069
>                 URL: https://issues.apache.org/jira/browse/FLINK-37069
>             Project: Flink
>          Issue Type: Sub-task
>            Reporter: Xintong Song
>            Assignee: Zakelly Lan
>            Priority: Blocker
>             Fix For: 2.0.0
>
>
> Instructions:
> First of all, please read the related documents briefly (still under review, will replace with formal links if merged):
> * Disaggregated State Management: https://github.com/apache/flink/pull/26107/files#diff-bfa19e04bb5c3487c3e9bf514d61c0fa8bb973950fb0ad0e3d4a6898a99b83e3
> * State V2: https://github.com/apache/flink/pull/26107/files#diff-5d1147987fecbda329132403c1d92384575be220092995c4be491e12b8c50cc9
> * ForSt State Backend: https://github.com/apache/flink/pull/26107/files#diff-b7c52c06f6ed4d5af6f230d11ba23ea051bf4a08c589d98392143f080c468a87
> For the SQL part, verification goes in FLINK-37068, we mainly focus on Datastream jobs and APIs here.
> 1. Make sure you are verifying this on master branch, since we have fixed several bugs since the rc0 package.
> 2. Choose one example in `flink-examples-streaming`. Most of the jobs has been rewritten using new API. Here we take `StateMachineExample` as an example.
> 3. Compile and run `StateMachineExample` in proper environment (I suggest a standalone session cluster or yarn), make sure you have the following command line params:
> {code:bash}
> ./flink run xxxxxxxxx \
>   --backend forst \
>   --checkpoint-dir s3://your/cp/dir \
>   --incremental-checkpoints true
> {code}
> Or set via `config.yaml`.
> {code:yaml}
> state.backend.type: forst
> execution.checkpointing.incremental: true
> execution.checkpointing.dir: s3://your-bucket/flink-checkpoints
> {code}
> 4. Check the job is running smoothly, the periodic checkpoints are successfully taken.
> 5. Stop the job and restart from the latest checkpoint.
> It would be great if you could write your own job using State V2 API, and follow the above Step 3~5. It is important to check whether there is any bug in new State APIs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)