[GitHub] [beam] scwhittle commented on a change in pull request #11289: [BEAM-9660]: Add an explicit check for integer overflow.

[GitBox <gi...@apache.org>]

scwhittle commented on a change in pull request #11289: [BEAM-9660]: Add an explicit check for integer overflow.
URL: https://github.com/apache/beam/pull/11289#discussion_r402466421
 
 

 ##########
 File path: runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
 ##########
 @@ -1372,11 +1372,13 @@ private void process(
       WorkItemCommitRequest commitRequest = outputBuilder.build();
       int byteLimit = maxWorkItemCommitBytes;
       int commitSize = commitRequest.getSerializedSize();
-      int estimatedCommitSize = commitSize < 0 ? Integer.MAX_VALUE : commitSize;
-
-      // Detect overflow of integer serialized size or if the byte limit was exceeded.
-      windmillMaxObservedWorkItemCommitBytes.addValue(estimatedCommitSize);
-      if (estimatedCommitSize > byteLimit) {
+      // Detect overflow of integer serialized size.
+      if (commitSize < 0) {
+        throw KeyCommitTooLargeException.causedBy(computationId, byteLimit, commitRequest);
 
 Review comment:
   We still would prefer to send a truncation commit (from buildWorkItemTruncationRequest).  That reports the commit as too large but sends the necessary info to the backend to attempt to truncate the work item for reprocessing.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services