You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/12/15 05:22:03 UTC
[GitHub] [flink] chucheng92 opened a new pull request, #21513: [FLINK-30424][streaming] Add source operator addSplits log when restore from state
chucheng92 opened a new pull request, #21513:
URL: https://github.com/apache/flink/pull/21513
## What is the purpose of the change
If source recover from state, we can not distinguish the newPartitions is from timed discover thread or from reader task state. So add recover log to help to debug and confirm this case.
## Brief change log
Add restore split log for SourceOperator.
## Verifying this change
Not add new cases. Just add log.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] chucheng92 commented on a diff in pull request #21513: [FLINK-30424][DataStream API] Add source operator addSplits log when restore from state
Posted by GitBox <gi...@apache.org>.
chucheng92 commented on code in PR #21513:
URL: https://github.com/apache/flink/pull/21513#discussion_r1049417560
##########
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/SourceOperator.java:
##########
@@ -333,6 +333,7 @@ public void open() throws Exception {
// restore the state if necessary.
final List<SplitT> splits = CollectionUtil.iterableToList(readerState.get());
if (!splits.isEmpty()) {
+ LOG.info("Restoring split(s) state to reader {}.", splits);
Review Comment:
@becketqin Hi, qin, i hava updated this pr, PTAL. thanks a lot.
##########
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/SourceOperator.java:
##########
@@ -333,6 +333,7 @@ public void open() throws Exception {
// restore the state if necessary.
final List<SplitT> splits = CollectionUtil.iterableToList(readerState.get());
if (!splits.isEmpty()) {
+ LOG.info("Restoring split(s) state to reader {}.", splits);
Review Comment:
@becketqin Hi, qin, i have updated this pr, PTAL. thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] chucheng92 commented on a diff in pull request #21513: [FLINK-30424][streaming] Add source operator addSplits log when restore from state
Posted by GitBox <gi...@apache.org>.
chucheng92 commented on code in PR #21513:
URL: https://github.com/apache/flink/pull/21513#discussion_r1049412147
##########
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/SourceOperator.java:
##########
@@ -333,6 +333,7 @@ public void open() throws Exception {
// restore the state if necessary.
final List<SplitT> splits = CollectionUtil.iterableToList(readerState.get());
if (!splits.isEmpty()) {
+ LOG.info("Restoring split(s) state to reader {}.", splits);
Review Comment:
> Because `sourceReader.addSplits(splits)` in the next line will log the details of the splits, maybe we can simply have `LOG.info("Restoring state for {} split(s) to reader.", splits.size())`?
yeah. i think u are right. because next SourceReaderBase will print adding splits details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] becketqin commented on pull request #21513: [FLINK-30424][DataStream API] Add source operator addSplits log when restore from state
Posted by GitBox <gi...@apache.org>.
becketqin commented on PR #21513:
URL: https://github.com/apache/flink/pull/21513#issuecomment-1355995629
Thanks for updating the patch. LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] becketqin commented on a diff in pull request #21513: [FLINK-30424][streaming] Add source operator addSplits log when restore from state
Posted by GitBox <gi...@apache.org>.
becketqin commented on code in PR #21513:
URL: https://github.com/apache/flink/pull/21513#discussion_r1049358274
##########
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/SourceOperator.java:
##########
@@ -333,6 +333,7 @@ public void open() throws Exception {
// restore the state if necessary.
final List<SplitT> splits = CollectionUtil.iterableToList(readerState.get());
if (!splits.isEmpty()) {
+ LOG.info("Restoring split(s) state to reader {}.", splits);
Review Comment:
Because `sourceReader.addSplits(splits)` in the next line will log the details of the splits, maybe we can simply have `LOG.info("Restoring state for {} split(s) to reader.", splits.size())`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #21513: [FLINK-30424][streaming] Add source operator addSplits log when restore from state
Posted by GitBox <gi...@apache.org>.
flinkbot commented on PR #21513:
URL: https://github.com/apache/flink/pull/21513#issuecomment-1352576672
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c8d4b98e6581e9c41a7aec3673a6073e40330285",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c8d4b98e6581e9c41a7aec3673a6073e40330285",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c8d4b98e6581e9c41a7aec3673a6073e40330285 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] becketqin merged pull request #21513: [FLINK-30424][DataStream API] Add source operator addSplits log when restore from state
Posted by GitBox <gi...@apache.org>.
becketqin merged PR #21513:
URL: https://github.com/apache/flink/pull/21513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] chucheng92 commented on a diff in pull request #21513: [FLINK-30424][streaming] Add source operator addSplits log when restore from state
Posted by GitBox <gi...@apache.org>.
chucheng92 commented on code in PR #21513:
URL: https://github.com/apache/flink/pull/21513#discussion_r1049417560
##########
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/SourceOperator.java:
##########
@@ -333,6 +333,7 @@ public void open() throws Exception {
// restore the state if necessary.
final List<SplitT> splits = CollectionUtil.iterableToList(readerState.get());
if (!splits.isEmpty()) {
+ LOG.info("Restoring split(s) state to reader {}.", splits);
Review Comment:
@becketqin Hi, qin, i updated this pr, PTAL. thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org