[GitHub] [spark] srielau commented on a diff in pull request #42564: [SPARK-44840][SQL] Make `array_insert()` 1-based for negative indexes

["srielau (via GitHub)" <gi...@apache.org>]

srielau commented on code in PR #42564:
URL: https://github.com/apache/spark/pull/42564#discussion_r1300174130


##########
docs/sql-migration-guide.md:
##########
@@ -28,6 +28,7 @@ license: |
 - Since Spark 3.5, Spark thrift server will interrupt task when canceling a running statement. To restore the previous behavior, set `spark.sql.thriftServer.interruptOnCancel` to `false`.
 - Since Spark 3.5, Row's json and prettyJson methods are moved to `ToJsonUtil`.
 - Since Spark 3.5, the `plan` field is moved from `AnalysisException` to `EnhancedAnalysisException`.
+- Since Spark 3.5, the `array_insert` function is 1-based for negative indexes. It inserts new element at the end of input arrays for the index -1. To restore the previous behavior, set `spark.sql.legacy.negativeIndexInArrayInsert` to `true`.

Review Comment:
   The behavior is inconsistent with itself.
   IFF array_insert() and (other functions) were 0-based then yes it would make sense that 1 is the second element and -1 the second last element.
   So to fix this function we either have to make it 0 based or change the behavior for negative indices.
   Against going 0-based speaks:
   1. 1-based was and remains a conscious decision
   2. We can agree (I think) that negative indices are less common as opposed to positive indices
   3. Our own docs actually don't have negative index examples and the instructions in the docs are ambiguous.
   
   Lastly the function is relatively new, the blast radius is limited.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org