You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "brijrajk (via GitHub)" <gi...@apache.org> on 2026/06/22 17:22:23 UTC

[PR] [SPARK-43322][SQL] Document null/empty behavior in explode_outer and posexplode_outer [spark]

brijrajk opened a new pull request, #56673:
URL: https://github.com/apache/spark/pull/56673

   ### What changes were proposed in this pull request?
   
   Update the `@ExpressionDescription` for `explode_outer` and `posexplode_outer` to explicitly state their behavior when the input array/map is null or empty, and add a null-input example to each.
   
   ### Why are the changes needed?
   
   The SQL function reference (generated from `@ExpressionDescription`) for `explode_outer` and `posexplode_outer` had identical `usage` text to their non-outer counterparts (`explode` / `posexplode`), omitting the defining difference: null or empty input produces a single null row rather than no rows. This information was already present in the PySpark docstrings but missing from the SQL reference.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No behavior change. Documentation only — the `DESCRIBE FUNCTION EXTENDED` output and generated SQL function reference now accurately describe the null/empty behavior.
   
   ### How was this patch tested?
   
   The catalyst module compiles cleanly. The example outputs were verified against the existing golden file `sql/core/src/test/resources/sql-tests/results/generators.sql.out`, which already tests `explode_outer(cast(null as array<int>))` and `posexplode_outer(cast(null as array<int>))` and confirms the expected `NULL` / `NULL\tNULL` output.
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: Claude Sonnet 4.6


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-43322][SQL] Document null/empty behavior in explode_outer and posexplode_outer [spark]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #56673:
URL: https://github.com/apache/spark/pull/56673#issuecomment-6243855515

   We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
   If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-43322][SQL] Document null/empty behavior in explode_outer and posexplode_outer [spark]

Posted by "brijrajk (via GitHub)" <gi...@apache.org>.
brijrajk commented on PR #56673:
URL: https://github.com/apache/spark/pull/56673#issuecomment-4785334411

   Good catch, thanks! The root issue is that `expressionGeneratorBuilderOuter("explode_outer", ExplodeExpressionBuilder)` in the `expressions` map takes the `@ExpressionDescription` from `ExplodeExpressionBuilder`, which is shared with `explode` and doesn't describe the null/empty behavior. I've added dedicated `ExplodeOuterExpressionBuilder` and `PosExplodeOuterExpressionBuilder` objects with the correct descriptions and updated the registrations.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-43322][SQL] Document null/empty behavior in explode_outer and posexplode_outer [spark]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #56673: [SPARK-43322][SQL] Document null/empty behavior in explode_outer and posexplode_outer
URL: https://github.com/apache/spark/pull/56673


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org