You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2021/09/01 09:27:46 UTC
[GitHub] [flink] lometheus commented on a change in pull request #17024: [FLINK-23457][network] The upstream sends the buffer of the right size for broadcast case
lometheus commented on a change in pull request #17024:
URL: https://github.com/apache/flink/pull/17024#discussion_r699305515
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/BufferWritingResultPartition.java
##########
@@ -330,9 +330,12 @@ private BufferBuilder appendBroadcastDataForRecordContinuation(
private void createBroadcastBufferConsumers(BufferBuilder buffer, int partialRecordBytes)
throws IOException {
try (final BufferConsumer consumer = buffer.createBufferConsumerFromBeginning()) {
+ int desirableBufferSize = Integer.MAX_VALUE;
for (ResultSubpartition subpartition : subpartitions) {
- subpartition.add(consumer.copy(), partialRecordBytes);
+ int subPartitionBufferSize = subpartition.add(consumer.copy(), partialRecordBytes);
Review comment:
Thank you very much for review. I add special treatment for negative value
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/BufferWritingResultPartition.java
##########
@@ -330,9 +330,12 @@ private BufferBuilder appendBroadcastDataForRecordContinuation(
private void createBroadcastBufferConsumers(BufferBuilder buffer, int partialRecordBytes)
throws IOException {
try (final BufferConsumer consumer = buffer.createBufferConsumerFromBeginning()) {
+ int desirableBufferSize = Integer.MAX_VALUE;
for (ResultSubpartition subpartition : subpartitions) {
- subpartition.add(consumer.copy(), partialRecordBytes);
+ int subPartitionBufferSize = subpartition.add(consumer.copy(), partialRecordBytes);
Review comment:
Thank you very much for review. I add special treatment for negative value
##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/ResultPartitionTest.java
##########
@@ -692,6 +692,17 @@ public void testDifferentBufferSizeForSubpartitions() throws IOException {
assertEquals(12, subpartition0.pollBuffer().buffer().getSize());
assertEquals(5, subpartition1.pollBuffer().buffer().getSize());
assertEquals(4, subpartition1.pollBuffer().buffer().getSize());
+
+ // when: Set the different buffers size.
+ subpartition0.bufferSize(12);
+ subpartition1.bufferSize(5);
+
+ // and: Add the buffer.
+ bufferWritingResultPartition.broadcastRecord(ByteBuffer.allocate(5));
+
+ // then: The buffer less or equal to configured.
+ assertEquals(5, subpartition0.pollBuffer().buffer().getSize());
+ assertEquals(5, subpartition1.pollBuffer().buffer().getSize());
Review comment:
I add testDynamicBufferSizeForBroadcast for less/greater/equal case
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org