[GitHub] zhijiangW commented on a change in pull request #6833: [FLINK-10537][network] Fix network small performance degradation after merging [FLINK-9913]

[GitBox <gi...@apache.org>]

zhijiangW commented on a change in pull request #6833: [FLINK-10537][network] Fix network small performance degradation after merging [FLINK-9913]
URL: https://github.com/apache/flink/pull/6833#discussion_r225030697
 
 

 ##########
 File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/serialization/SpanningRecordSerializer.java
 ##########
 @@ -92,20 +92,9 @@ public void serializeRecord(T record) throws IOException {
 	 */
 	@Override
 	public SerializationResult copyToBufferBuilder(BufferBuilder targetBuffer) {
-		boolean mustCommit = false;
-		if (lengthBuffer.hasRemaining()) {
 
 Review comment:
   It is really not necessary to check the conditions for every record in normal cases, and it may bring some overheads to do so.
   
   But if we directly remove these conditions, in the case of spanning multiple buffers because of large records or remaining small spaces in previous `targetBuffer`, we may encounter another cost overhead. In this condition, the length buffer is probably not remaining but only the data buffer is remaining, and the cost of `targetBuffer.append(lengthBuffer)` would be higher than the cost of `lengthBuffer.hasRemaining()`.
   
   If we can check this condition on `RecordWriter` side as the following, maybe we can check accordingly in `RecordSerializer`.
   
   ```
   private boolean copyFromSerializerToTargetChannel(int targetChannel)  {
   		serializer.reset();
   
   		boolean pruneTriggered = false;
   		BufferBuilder bufferBuilder = getBufferBuilder(targetChannel);
   		SerializationResult result = serializer.copyToBufferBuilder(bufferBuilder, **false**);
   		while (result.isFullBuffer()) {
   			numBytesOut.inc(bufferBuilder.finish());
   			numBuffersOut.inc();
   
   			if (result.isFullRecord()) {
   				pruneTriggered = true;
   				bufferBuilders[targetChannel] = Optional.empty();
   				break;
   			}
   
   			bufferBuilder = requestNewBufferBuilder(targetChannel);
   			result = serializer.copyToBufferBuilder(bufferBuilder, **true**);
   		}
   ```
   
   But I am not sure whether it is worth doing by giving this hint in `copyToBufferBuilder` interface. What do you think?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services