You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Danny Cranmer (Jira)" <ji...@apache.org> on 2020/07/07 07:47:00 UTC
[jira] [Updated] (FLINK-18512) [KINESIS][EFO] Introduce
RecordPublisher Interface
[ https://issues.apache.org/jira/browse/FLINK-18512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Danny Cranmer updated FLINK-18512:
----------------------------------
Description:
*Background*
In order to add support for EFO in the {{FlinkKinesisConsumer}} we are abstracting out the record consumption from Kinesis {{ShardConsumer}} and introducing an interface.
*Scope*
Introduce the {{RecordPublisher}} interface and refactor the existing polling implementation to implement it:
* Add {{PollingRecordPublisher}} that is functionally equivalent to the existing implementation
* Support adaptive throughput via an extension, {{AdaptivePollingRecordSubscriber}}
* Split out the {{ShardMetricReporter}} into separate classes such that each component can report it's own metrics:
** {{ShardConsumer}}
** {{PollingRecordConsumer}}
** {{FanOutRecordConsumer}} (later)
* All the existing unit test will continue to pass, and be functionally equivalent (there may be minor compilation tweaks)
was:
*Background*
In order to add support for EFO in the {{FlinkKinesisConsumer}} we are abstracting out the record consumption from Kinesis {{ShardConsumer}} and introducing an interface.
*Scope*
Introduce the {{RecordPublisher}} interface and refactor the existing polling implementation to implement it:
* Add {{PollingRecordPublisher}} that is functional equivalent to the existing implementation
* Support adaptive throughput via an extension, {{AdaptivePollingRecordSubscriber}}
* Split out the {{ShardMetricReporter}} into separate classes such that each component can report it's own metrics:
** {{ShardConsumer}}
** {{PollingRecordConsumer}}
** {{FanOutRecordConsumer}} (later)
* All the existing unit test will continue to pass, and be functionally equivalent (there may be minor compilation tweaks)
> [KINESIS][EFO] Introduce RecordPublisher Interface
> --------------------------------------------------
>
> Key: FLINK-18512
> URL: https://issues.apache.org/jira/browse/FLINK-18512
> Project: Flink
> Issue Type: Sub-task
> Reporter: Danny Cranmer
> Priority: Major
>
> *Background*
> In order to add support for EFO in the {{FlinkKinesisConsumer}} we are abstracting out the record consumption from Kinesis {{ShardConsumer}} and introducing an interface.
> *Scope*
> Introduce the {{RecordPublisher}} interface and refactor the existing polling implementation to implement it:
> * Add {{PollingRecordPublisher}} that is functionally equivalent to the existing implementation
> * Support adaptive throughput via an extension, {{AdaptivePollingRecordSubscriber}}
> * Split out the {{ShardMetricReporter}} into separate classes such that each component can report it's own metrics:
> ** {{ShardConsumer}}
> ** {{PollingRecordConsumer}}
> ** {{FanOutRecordConsumer}} (later)
> * All the existing unit test will continue to pass, and be functionally equivalent (there may be minor compilation tweaks)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)