You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2019/04/05 09:32:01 UTC
[GitHub] [flink] azagrebin commented on issue #7835: [FLINK-11750][network]
Replace IntermediateResultPartitionID with ResultPartitionID in
ResultPartitionDeploymentDescriptor
azagrebin commented on issue #7835: [FLINK-11750][network] Replace IntermediateResultPartitionID with ResultPartitionID in ResultPartitionDeploymentDescriptor
URL: https://github.com/apache/flink/pull/7835#issuecomment-480211309
@zhijiangW , thinking about [result partition lifecycle](https://docs.google.com/document/d/13vAJJxfRXAwI4MtO8dux8hHnNMw2Biu5XRrb_hvGehA/edit#heading=h.v7vhb7w01d61), I am not sure that `ResultPartitionDeploymentDescriptor` should always have `ExecutionAttemptID`.
If we assume there is a case when consumer is deployed without producer to consume already previously finished partition then JM might not even know previous `ExecutionAttemptID`. The workflow in this case could be though that JM did previously set `ExecutionAttemptID` of deployed producer in `ProducerShuffleDescriptor`. The ShuffleMaster which supports external partitions could store this `ExecutionAttemptID` with finished partition and put into `ShuffleDeploymentDescriptor` if needed in both cases: `ShuffleMaster.getPartition` (stored previously) and `ShuffleMaster.allocateNewPartition` (from `ProducerShuffleDescriptor`).
At the moment, AFAIK, we use this `ExecutionAttemptID` only internally in `NetworkEnviroment` for indexing and verification of requested by consumer partition in producer.
What do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services