[GitHub] NicoK commented on a change in pull request #6705: [FLINK-10356][network] add sanity checks to SpillingAdaptiveSpanningRecordDeserializer

[GitBox <gi...@apache.org>]

NicoK commented on a change in pull request #6705: [FLINK-10356][network] add sanity checks to SpillingAdaptiveSpanningRecordDeserializer
URL: https://github.com/apache/flink/pull/6705#discussion_r226979613
 
 

 ##########
 File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/serialization/SpillingAdaptiveSpanningRecordDeserializer.java
 ##########
 @@ -549,21 +584,53 @@ private void addNextChunkFromMemorySegment(MemorySegment segment, int offset, in
 				}
 				else {
 					spillingChannel.close();
+					spillingChannel = null;
 
-					BufferedInputStream inStream = new BufferedInputStream(new FileInputStream(spillFile), 2 * 1024 * 1024);
+					BufferedInputStream inStream =
+						new BufferedInputStream(
+							new FileInputStream(checkNotNull(spillFile)),
+							2 * 1024 * 1024);
 					this.spillFileReader = new DataInputViewStreamWrapper(inStream);
 				}
 			}
 		}
 
-		private void moveRemainderToNonSpanningDeserializer(NonSpanningWrapper deserializer) {
+		private void moveRemainderToNonSpanningDeserializer(NonSpanningWrapper deserializer) throws IOException {
+			Optional<String> deserializationError = getDeserializationError(0);
+			if (deserializationError.isPresent()) {
+				throw new IOException(deserializationError.get());
+			}
+
 			deserializer.clear();
 
 			if (leftOverData != null) {
 				deserializer.initializeFromMemorySegment(leftOverData, leftOverStart, leftOverLimit);
 			}
 		}
 
+		private Optional<String> getDeserializationError(int addToReadBytes) {
 
 Review comment:
   It took me a bit, but then I realised why: If this code throws an `EOFException`, `remainingSpanningBytes` will report `0` and not fail (and fall back to the thrown `EOFException` instead of our more detailed custom one):
   ```
   			try {
   				target.read(this.spanningWrapper.getInputView());
   			} catch (EOFException e) {
   ```
   -> by simulating that we read too many bytes (which is actually true), we get our exception containing the line `-1 remaining unread byte`. It is a bit hacky though. Let me try to refactor and add comments nonetheless.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services