[GitHub] [spark] cloud-fan commented on a diff in pull request #37852: [SPARK-40403][SQL] Calculate unsafe array size using longs to avoid negative size in error message

[GitBox <gi...@apache.org>]

cloud-fan commented on code in PR #37852:
URL: https://github.com/apache/spark/pull/37852#discussion_r969142012


##########
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeArrayWriter.java:
##########
@@ -55,10 +55,19 @@ public void initialize(int numElements) {
 
     this.startingOffset = cursor();
 
+    long fixedPartInBytesLong =
+      ByteArrayMethods.roundNumberOfBytesToNearestWord((long) elementSize * numElements);
+    long totalInitialSize = headerInBytes + fixedPartInBytesLong;
+
+    if (totalInitialSize > Integer.MAX_VALUE) {
+      throw new IllegalArgumentException(

Review Comment:
   I think so, let's make the error more user-facing.
   error class: TOO_MANY_ARRAY_ELEMENTS
   message: "Cannot initialize array with %numElements elements"



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org