You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2021/01/04 09:11:57 UTC
[GitHub] [flink] curcur opened a new pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
curcur opened a new pull request #14546:
URL: https://github.com/apache/flink/pull/14546
… (AC/UC) for each subtask
## What is the purpose of the change
In the UI, show checkpoint type for each subtask (add an unaligned checkpoint column for each subtask).
## Brief change log
## Verifying this change
Existing Unit test and also tested on web-UI
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] curcur commented on pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
curcur commented on pull request #14546:
URL: https://github.com/apache/flink/pull/14546#issuecomment-755046097
https://dev.azure.com/mymeiyuan/Flink/_build/results?buildId=152&view=results
local azure is green
the cluster azure fails with unrelated tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] curcur commented on pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
curcur commented on pull request #14546:
URL: https://github.com/apache/flink/pull/14546#issuecomment-754495311
> The changes generally LGTM but not to spotless :) Please fix formatting.
> I ran a test job and I think aggregation can be added later as a separate PR.
hoops, forget to do the formatting ...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] curcur commented on pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
curcur commented on pull request #14546:
URL: https://github.com/apache/flink/pull/14546#issuecomment-753856508
https://files.slack.com/files-pri/T0763FK1B-F01HMB78WAJ/_____________2020-12-31_______3.53.55.png
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14546:
URL: https://github.com/apache/flink/pull/14546#issuecomment-753856824
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit beada5bb875badb0383e2198b5c7677ea2193281 (Fri May 28 07:02:34 UTC 2021)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] curcur commented on a change in pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
curcur commented on a change in pull request #14546:
URL: https://github.com/apache/flink/pull/14546#discussion_r551859021
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/SubtaskCheckpointCoordinatorImpl.java
##########
@@ -297,6 +297,11 @@ public void checkpointState(
try {
if (takeSnapshotSync(
snapshotFutures, metadata, metrics, options, operatorChain, isCanceled)) {
+ LOG.debug(
+ "Is unaligned checkpoint for task {} : {}",
Review comment:
I moved both the log and metrics update to the end of `takeSnapshotSync `.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14546:
URL: https://github.com/apache/flink/pull/14546#issuecomment-753861895
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3c05ee316221c4cd8e58ff36de115c7c759d8e39",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11597",
"triggerID" : "3c05ee316221c4cd8e58ff36de115c7c759d8e39",
"triggerType" : "PUSH"
}, {
"hash" : "beada5bb875badb0383e2198b5c7677ea2193281",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "beada5bb875badb0383e2198b5c7677ea2193281",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3c05ee316221c4cd8e58ff36de115c7c759d8e39 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11597)
* beada5bb875badb0383e2198b5c7677ea2193281 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rkhachatryan commented on a change in pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
rkhachatryan commented on a change in pull request #14546:
URL: https://github.com/apache/flink/pull/14546#discussion_r551614695
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/SubtaskCheckpointCoordinatorImpl.java
##########
@@ -297,6 +297,11 @@ public void checkpointState(
try {
if (takeSnapshotSync(
snapshotFutures, metadata, metrics, options, operatorChain, isCanceled)) {
+ LOG.debug(
+ "Is unaligned checkpoint for task {} : {}",
Review comment:
I think a better place to log is the end of `takeSnapshotSync` where other properties are logged.
##########
File path: flink-runtime-web/web-dashboard/src/app/pages/job/checkpoints/subtask/job-checkpoints-subtask.component.html
##########
@@ -91,6 +91,7 @@
<th nzSortKey="alignment.processed" nzShowSort><strong>Processed (persisted) Data</strong></th>
<th nzSortKey="alignment.duration" nzShowSort><strong>Alignment Duration</strong></th>
<th nzSortKey="start_delay" nzShowSort><strong>Start Delay</strong></th>
+ <th nzSortKey="unaligned_checkpoint" nzShowSort><strong>Unaligned Checkpoint</strong></th>
Review comment:
nit: invert to Aligned to save space?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14546:
URL: https://github.com/apache/flink/pull/14546#issuecomment-753861895
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3c05ee316221c4cd8e58ff36de115c7c759d8e39",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11597",
"triggerID" : "3c05ee316221c4cd8e58ff36de115c7c759d8e39",
"triggerType" : "PUSH"
}, {
"hash" : "beada5bb875badb0383e2198b5c7677ea2193281",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11652",
"triggerID" : "beada5bb875badb0383e2198b5c7677ea2193281",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* beada5bb875badb0383e2198b5c7677ea2193281 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #14546:
URL: https://github.com/apache/flink/pull/14546#issuecomment-753861895
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3c05ee316221c4cd8e58ff36de115c7c759d8e39",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3c05ee316221c4cd8e58ff36de115c7c759d8e39",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3c05ee316221c4cd8e58ff36de115c7c759d8e39 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] curcur commented on pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
curcur commented on pull request #14546:
URL: https://github.com/apache/flink/pull/14546#issuecomment-754567585
I re-tested after updates.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] curcur commented on a change in pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
curcur commented on a change in pull request #14546:
URL: https://github.com/apache/flink/pull/14546#discussion_r551790228
##########
File path: flink-runtime-web/web-dashboard/src/app/pages/job/checkpoints/subtask/job-checkpoints-subtask.component.html
##########
@@ -91,6 +91,7 @@
<th nzSortKey="alignment.processed" nzShowSort><strong>Processed (persisted) Data</strong></th>
<th nzSortKey="alignment.duration" nzShowSort><strong>Alignment Duration</strong></th>
<th nzSortKey="start_delay" nzShowSort><strong>Start Delay</strong></th>
+ <th nzSortKey="unaligned_checkpoint" nzShowSort><strong>Unaligned Checkpoint</strong></th>
Review comment:
you sure...? The meaning of "Aligned" is ambiguous, and I do not think it is straightforward to users that "Aligned" means "not completed as an unaligned checkpoint"?
##########
File path: flink-runtime-web/web-dashboard/src/app/pages/job/checkpoints/subtask/job-checkpoints-subtask.component.html
##########
@@ -91,6 +91,7 @@
<th nzSortKey="alignment.processed" nzShowSort><strong>Processed (persisted) Data</strong></th>
<th nzSortKey="alignment.duration" nzShowSort><strong>Alignment Duration</strong></th>
<th nzSortKey="start_delay" nzShowSort><strong>Start Delay</strong></th>
+ <th nzSortKey="unaligned_checkpoint" nzShowSort><strong>Unaligned Checkpoint</strong></th>
Review comment:
you sure...? The meaning of "Aligned" is ambiguous, and I do not think it is straightforward for users that "Aligned" means "not completed as an unaligned checkpoint"?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14546:
URL: https://github.com/apache/flink/pull/14546#issuecomment-753861895
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3c05ee316221c4cd8e58ff36de115c7c759d8e39",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11597",
"triggerID" : "3c05ee316221c4cd8e58ff36de115c7c759d8e39",
"triggerType" : "PUSH"
}, {
"hash" : "beada5bb875badb0383e2198b5c7677ea2193281",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11652",
"triggerID" : "beada5bb875badb0383e2198b5c7677ea2193281",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3c05ee316221c4cd8e58ff36de115c7c759d8e39 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11597)
* beada5bb875badb0383e2198b5c7677ea2193281 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rkhachatryan commented on a change in pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
rkhachatryan commented on a change in pull request #14546:
URL: https://github.com/apache/flink/pull/14546#discussion_r551808435
##########
File path: flink-runtime-web/web-dashboard/src/app/pages/job/checkpoints/subtask/job-checkpoints-subtask.component.html
##########
@@ -91,6 +91,7 @@
<th nzSortKey="alignment.processed" nzShowSort><strong>Processed (persisted) Data</strong></th>
<th nzSortKey="alignment.duration" nzShowSort><strong>Alignment Duration</strong></th>
<th nzSortKey="start_delay" nzShowSort><strong>Start Delay</strong></th>
+ <th nzSortKey="unaligned_checkpoint" nzShowSort><strong>Unaligned Checkpoint</strong></th>
Review comment:
We could add a tooltip (and it should be documented).
I'm also fine with "Unaligned Checkpoint".
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] curcur removed a comment on pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
curcur removed a comment on pull request #14546:
URL: https://github.com/apache/flink/pull/14546#issuecomment-753856508
https://files.slack.com/files-pri/T0763FK1B-F01HMB78WAJ/_____________2020-12-31_______3.53.55.png
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14546:
URL: https://github.com/apache/flink/pull/14546#issuecomment-753861895
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3c05ee316221c4cd8e58ff36de115c7c759d8e39",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11597",
"triggerID" : "3c05ee316221c4cd8e58ff36de115c7c759d8e39",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3c05ee316221c4cd8e58ff36de115c7c759d8e39 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11597)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #14546:
URL: https://github.com/apache/flink/pull/14546#issuecomment-753856824
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 3c05ee316221c4cd8e58ff36de115c7c759d8e39 (Mon Jan 04 09:14:21 UTC 2021)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
* **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-20488).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work.
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pnowojski merged pull request #14546: [FLINK-20488][runtime][checkpoint] Show checkpoint type in the Web-UI…
Posted by GitBox <gi...@apache.org>.
pnowojski merged pull request #14546:
URL: https://github.com/apache/flink/pull/14546
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org