[I] [Feature] Support Spark expression: multiply_ym_interval [datafusion-comet]

["andygrove (via GitHub)" <gi...@apache.org>]

andygrove opened a new issue, #3102:
URL: https://github.com/apache/datafusion-comet/issues/3102

   ## What is the problem the feature request solves?
   
   > **Note:** This issue was generated with AI assistance. The specification details have been extracted from Spark documentation and may need verification.
   
   Comet does not currently support the Spark `multiply_ym_interval` function, causing queries using this function to fall back to Spark's JVM execution instead of running natively on DataFusion.
   
   The `MultiplyYMInterval` expression multiplies a year-month interval by a numeric value, returning a new year-month interval. This expression supports various numeric data types and handles precision conversions appropriately for each type.
   
   Supporting this expression would allow more Spark workloads to benefit from Comet's native acceleration.
   
   ## Describe the potential solution
   
   ### Spark Specification
   
   **Syntax:**
   ```sql
   year_month_interval * numeric_value
   ```
   
   **Arguments:**
   | Argument | Type | Description |
   |----------|------|-------------|
   | `interval` | YearMonthIntervalType | The year-month interval to be multiplied |
   | `num` | NumericType | The numeric multiplier (byte, short, int, long, float, double, or decimal) |
   
   **Return Type:** `YearMonthIntervalType()` - Returns a year-month interval representing the multiplication result.
   
   **Supported Data Types:**
   - **Interval**: YearMonthIntervalType only
   - **Numeric**: All numeric types including:
     - Integer types: ByteType, ShortType, IntegerType, LongType
     - Floating-point types: FloatType, DoubleType
     - DecimalType (arbitrary precision)
   
   **Edge Cases:**
   - **Null handling**: Returns null if either operand is null (nullIntolerant = true)
   - **Overflow behavior**: 
     - Integer multiplication uses `Math.multiplyExact()` throwing ArithmeticException on overflow
     - Long results converted using `Math.toIntExact()` throwing exception if out of int range
     - Decimal results use `intValueExact()` throwing exception if fractional part exists after rounding
   - **Floating-point precision**: Uses HALF_UP rounding mode for consistent behavior
   - **Zero multiplication**: Results in zero-month interval (valid)
   
   **Examples:**
   ```sql
   -- Multiply interval by integer
   SELECT INTERVAL '2-6' YEAR TO MONTH * 3;  -- Results in '7-6' (7 years 6 months)
   
   -- Multiply interval by decimal
   SELECT INTERVAL '1-0' YEAR TO MONTH * 2.5;  -- Results in '2-6' (2 years 6 months)
   ```
   
   ```scala
   // DataFrame API usage
   import org.apache.spark.sql.functions._
   
   df.select(col("year_month_interval") * lit(2))
   df.select(col("year_month_interval") * col("multiplier"))
   ```
   
   ### Implementation Approach
   
   See the [Comet guide on adding new expressions](https://datafusion.apache.org/comet/contributor-guide/adding_a_new_expression.html) for detailed instructions.
   
   1. **Scala Serde**: Add expression handler in `spark/src/main/scala/org/apache/comet/serde/`
   2. **Register**: Add to appropriate map in `QueryPlanSerde.scala`
   3. **Protobuf**: Add message type in `native/proto/src/proto/expr.proto` if needed
   4. **Rust**: Implement in `native/spark-expr/src/` (check if DataFusion has built-in support first)
   
   
   ## Additional context
   
   **Difficulty:** Medium
   **Spark Expression Class:** `org.apache.spark.sql.catalyst.expressions.MultiplyYMInterval`
   
   **Related:**
   - `MultiplyDTInterval` - For day-time interval multiplication
   - `DivideYMInterval` - For year-month interval division
   - `AddYMInterval` - For year-month interval addition
   - `SubtractYMInterval` - For year-month interval subtraction
   
   ---
   *This issue was auto-generated from Spark reference documentation.*
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] [Feature] Support Spark expression: multiply_ym_interval [datafusion-comet]

["peterxcli (via GitHub)" <gi...@apache.org>]

peterxcli commented on issue #3102:
URL: https://github.com/apache/datafusion-comet/issues/3102#issuecomment-4950499362

   take


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] [Feature] Support Spark expression: multiply_ym_interval [datafusion-comet]

["sunchao (via GitHub)" <gi...@apache.org>]

sunchao closed issue #3102: [Feature] Support Spark expression: multiply_ym_interval
URL: https://github.com/apache/datafusion-comet/issues/3102


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org