[I] [Feature] Support Spark expression: next_day [datafusion-comet]

["andygrove (via GitHub)" <gi...@apache.org>]

andygrove opened a new issue, #3092:
URL: https://github.com/apache/datafusion-comet/issues/3092

   ## What is the problem the feature request solves?
   
   > **Note:** This issue was generated with AI assistance. The specification details have been extracted from Spark documentation and may need verification.
   
   Comet does not currently support the Spark `next_day` function, causing queries using this function to fall back to Spark's JVM execution instead of running natively on DataFusion.
   
   The `NextDay` expression returns the first date after a given start date that falls on a specified day of the week. It advances from the start date to find the next occurrence of the target day, excluding the start date itself even if it matches the target day of week.
   
   Supporting this expression would allow more Spark workloads to benefit from Comet's native acceleration.
   
   ## Describe the potential solution
   
   ### Spark Specification
   
   **Syntax:**
   ```sql
   next_day(start_date, day_of_week)
   ```
   
   ```scala
   // DataFrame API
   col("date_column").next_day("Monday")
   next_day(col("start_date"), col("day_name"))
   ```
   
   **Arguments:**
   | Argument | Type | Description |
   |----------|------|-------------|
   | startDate | DateType | The starting date from which to find the next occurrence |
   | dayOfWeek | StringType (with collation support) | The target day of week as a string (e.g., "Monday", "Tue") |
   | failOnError | Boolean | Internal parameter controlling ANSI mode behavior for invalid inputs |
   
   **Return Type:** Returns `DateType` - an integer representing days since epoch (1970-01-01).
   
   **Supported Data Types:**
   - **Input**: DateType for start date, StringType with collation support for day of week
   - **Output**: DateType
   - **Trimming**: Supports trim collation for the day of week string parameter
   
   **Edge Cases:**
   - **Null handling**: Returns null if either input is null (null intolerant behavior)
   - **Invalid day names**: Throws `SparkIllegalArgumentException` in ANSI mode, returns null otherwise
   - **Case sensitivity**: Day of week parsing follows `DateTimeUtils` case handling rules
   - **Abbreviations**: Supports abbreviated day names (implementation dependent on `DateTimeUtils`)
   - **Same day exclusion**: Never returns the start date itself, always advances to next occurrence
   
   **Examples:**
   ```sql
   -- Find next Monday after January 1st, 2023
   SELECT next_day('2023-01-01', 'Monday');
   -- Returns: 2023-01-02
   
   -- Using with column references
   SELECT order_date, next_day(order_date, 'Friday') as next_friday
   FROM orders;
   
   -- Next Tuesday after current date
   SELECT next_day(current_date(), 'Tue');
   ```
   
   ```scala
   // DataFrame API examples
   import org.apache.spark.sql.functions._
   
   // Find next Monday for each date
   df.select(col("start_date"), next_day(col("start_date"), lit("Monday")))
   
   // Dynamic day of week from another column  
   df.select(next_day(col("event_date"), col("target_day")))
   
   // Using string interpolation
   df.withColumn("next_sunday", next_day($"date_col", "Sunday"))
   ```
   
   ### Implementation Approach
   
   See the [Comet guide on adding new expressions](https://datafusion.apache.org/comet/contributor-guide/adding_a_new_expression.html) for detailed instructions.
   
   1. **Scala Serde**: Add expression handler in `spark/src/main/scala/org/apache/comet/serde/`
   2. **Register**: Add to appropriate map in `QueryPlanSerde.scala`
   3. **Protobuf**: Add message type in `native/proto/src/proto/expr.proto` if needed
   4. **Rust**: Implement in `native/spark-expr/src/` (check if DataFusion has built-in support first)
   
   
   ## Additional context
   
   **Difficulty:** Medium
   **Spark Expression Class:** `org.apache.spark.sql.catalyst.expressions.NextDay`
   
   **Related:**
   - `date_add` - Add days to a date
   - `date_sub` - Subtract days from a date  
   - `dayofweek` - Extract day of week from date
   - `last_day` - Get last day of month
   - `DateTimeUtils` - Underlying utility class for date operations
   
   ---
   *This issue was auto-generated from Spark reference documentation.*
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] [Feature] Support Spark expression: next_day [datafusion-comet]

["mbutrovich (via GitHub)" <gi...@apache.org>]

mbutrovich closed issue #3092: [Feature] Support Spark expression: next_day
URL: https://github.com/apache/datafusion-comet/issues/3092


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org