You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/07/12 18:10:58 UTC
[GitHub] [flink] Cyberness opened a new pull request, #20257: [FLINK-24943][Connectors / Kinesis] Explicitly create KryoSerializer for SequenceNumber class in Kinesis Consumer
Cyberness opened a new pull request, #20257:
URL: https://github.com/apache/flink/pull/20257
### What is the purpose of the change
Create `KryoSerializer` for `SequenceNumber` class explicitly to allow disabling of Generic Types via disableGenericTypes() call.
### Brief change log
1. Create KinesisStateUtil class with static method `createShardsStateSerializer` that explicitly assigns `KryoSerializer` to `SequenceNumber` class and use this method to initialize state
2. Create unit test that checks for compatibility of previous `TypeInformation` based serializer and explicitly created `KryoSerializer`
3. Replace `Mokito` calls with utility `MockStreamingRuntimeContext` to improve testability and unify initialization method of Streaming Runtime Context
### Verifying this change
Added unit test to verify changed initialization of state.
### Does this pull request potentially affect one of the following parts:
Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no
### Documentation
Does this pull request introduce a new feature?: no
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented): no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] Cyberness commented on pull request #20257: [FLINK-24943][Connectors / Kinesis] Explicitly create KryoSerializer for SequenceNumber class in Kinesis Consumer
Posted by GitBox <gi...@apache.org>.
Cyberness commented on PR #20257:
URL: https://github.com/apache/flink/pull/20257#issuecomment-1275379856
> The code changes look good. Recognise that we specify a KryoSerializer instead of making the SequenceNumber a POJO to maintain backwards compatibility. Given that we are changing the state serializer, could we do a sanity test to check the following?
>
> * we can disableGenericTypes (as reported [here](https://issues.apache.org/jira/browse/FLINK-24549))
> * a snapshot created from old consumer is valid for a new consumer
Yes, I can work on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] hlteoh37 commented on pull request #20257: [FLINK-24943][Connectors / Kinesis] Explicitly create KryoSerializer for SequenceNumber class in Kinesis Consumer
Posted by GitBox <gi...@apache.org>.
hlteoh37 commented on PR #20257:
URL: https://github.com/apache/flink/pull/20257#issuecomment-1275342620
The code changes look good. Recognise that we specify a KryoSerializer instead of making the SequenceNumber a POJO to maintain backwards compatibility. Given that we are changing the state serializer, could we do a sanity test to check the following?
- we can disableGenericTypes (as reported [here](https://issues.apache.org/jira/browse/FLINK-24549))
- a snapshot created from old consumer is valid for a new consumer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] Cyberness closed pull request #20257: [FLINK-24943][Connectors / Kinesis] Explicitly create KryoSerializer for SequenceNumber class in Kinesis Consumer
Posted by GitBox <gi...@apache.org>.
Cyberness closed pull request #20257: [FLINK-24943][Connectors / Kinesis] Explicitly create KryoSerializer for SequenceNumber class in Kinesis Consumer
URL: https://github.com/apache/flink/pull/20257
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-24943][Connectors / Kinesis] Explicitly create KryoSerializer for SequenceNumber class in Kinesis Consumer [flink]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20257:
URL: https://github.com/apache/flink/pull/20257#issuecomment-2768298817
This PR has been closed since it has not had any activity in 120 days.
If you feel like this was a mistake, or you would like to continue working on it,
please feel free to re-open the PR and ask for a review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] Cyberness commented on pull request #20257: [FLINK-24943][Connectors / Kinesis] Explicitly create KryoSerializer for SequenceNumber class in Kinesis Consumer
Posted by GitBox <gi...@apache.org>.
Cyberness commented on PR #20257:
URL: https://github.com/apache/flink/pull/20257#issuecomment-1379570234
Does it still make sense to work on merging this change considering [FLINK-30557](https://github.com/apache/flink/pull/21600)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dannycranmer commented on pull request #20257: [FLINK-24943][Connectors / Kinesis] Explicitly create KryoSerializer for SequenceNumber class in Kinesis Consumer
Posted by GitBox <gi...@apache.org>.
dannycranmer commented on PR #20257:
URL: https://github.com/apache/flink/pull/20257#issuecomment-1279205494
@Cyberness please also squash commits and update commit message as per the [contribution guide](https://flink.apache.org/contributing/contribute-documentation.html#submit-your-contribution)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-24943][Connectors / Kinesis] Explicitly create KryoSerializer for SequenceNumber class in Kinesis Consumer [flink]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #20257: [FLINK-24943][Connectors / Kinesis] Explicitly create KryoSerializer for SequenceNumber class in Kinesis Consumer
URL: https://github.com/apache/flink/pull/20257
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #20257: [FLINK-24943][Connectors / Kinesis] Explicitly create KryoSerializer for SequenceNumber class in Kinesis Consumer
Posted by GitBox <gi...@apache.org>.
flinkbot commented on PR #20257:
URL: https://github.com/apache/flink/pull/20257#issuecomment-1182178961
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6de30f8d2e3b669acc844ace806eb1c2eaf319b2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6de30f8d2e3b669acc844ace806eb1c2eaf319b2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6de30f8d2e3b669acc844ace806eb1c2eaf319b2 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] hlteoh37 commented on pull request #20257: [FLINK-24943][Connectors / Kinesis] Explicitly create KryoSerializer for SequenceNumber class in Kinesis Consumer
Posted by GitBox <gi...@apache.org>.
hlteoh37 commented on PR #20257:
URL: https://github.com/apache/flink/pull/20257#issuecomment-1379661435
We have now moved the consumer here https://github.com/apache/flink-connector-aws
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-24943][Connectors / Kinesis] Explicitly create KryoSerializer for SequenceNumber class in Kinesis Consumer [flink]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20257:
URL: https://github.com/apache/flink/pull/20257#issuecomment-2592595579
This PR is being marked as stale since it has not had any activity in the last 180 days.
If you would like to keep this PR alive, please leave a comment asking for a review.
If the PR has merge conflicts, update it with the latest from the base branch.
<p>
If you are having difficulty finding a reviewer, please reach out to the
[community](https://flink.apache.org/what-is-flink/community/).
<p>
If this PR is no longer valid or desired, please feel free to close it.
If no activity occurs in the next 90 days, it will be automatically closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org