You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Sahitya0805 (via GitHub)" <gi...@apache.org> on 2026/02/09 09:26:03 UTC

[PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Sahitya0805 opened a new pull request, #20229:
URL: https://github.com/apache/datafusion/pull/20229

   ## Which issue does this PR close?
   
   Closes #20227
   
   ## Rationale for this change
   
   The current `Expr::sort(bool, bool)` API suffers from "boolean blindness" - it's not clear what `sort(true, false)` means without checking the documentation.
   
   ## What changes are included in this PR?
   
   - Added `SortOptions` struct with fluent builder methods (`desc()`, `asc()`, `nulls_first()`, `nulls_last()`)
   - Added `Expr::sort_by()` method that accepts `SortOptions`
   - Kept existing `Expr::sort()` method for backward compatibility
   - Added test demonstrating both old and new APIs
   
   ## Are these changes tested?
   
   Yes, added `repro_sort_api.rs` test that verifies:
   - Old API still works correctly
   - New API produces correct `Sort` struct
   - Default options work as expected
   
   ## Are there any user-facing changes?
   
   Yes, users can now use the more readable API:
   ```rust
   // Old API (still supported)
   col("foo").sort(true, false)
   
   // New API (recommended)
   col("foo").sort_by(SortOptions::new().desc().nulls_first())
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "Sahitya0805 (via GitHub)" <gi...@apache.org>.
Sahitya0805 commented on PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#issuecomment-3908758676

   Just following up on the PR I raised.
   Looking forward to your feedback. Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "Banyc (via GitHub)" <gi...@apache.org>.
Banyc commented on code in PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#discussion_r2781681022


##########
datafusion/expr/src/sort_options.rs:
##########
@@ -0,0 +1,50 @@
+use arrow::compute::SortOptions as ArrowSortOptions;
+
+/// Options for sorting.
+/// 
+/// This struct implements a builder pattern for creating `SortOptions`.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
+pub struct SortOptions {
+    pub descending: bool,
+    pub nulls_first: bool,
+}
+
+impl SortOptions {
+    /// Create a new `SortOptions` with default values (Ascending, Nulls Last).
+    pub fn new() -> Self {
+        Self::default()
+    }

Review Comment:
   clankers be like: wow builder pattern yay



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "Banyc (via GitHub)" <gi...@apache.org>.
Banyc commented on PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#issuecomment-3875095005

   @adriangb 
   - I am upset by the poor code quality here:
      ```rust
      // my comment for this: ...this is again hiding information from use sites...
      pub fn new() -> Self {
         Self::default()
      }
      ```
      to 
      ```rust
      pub fn new() -> Self {
           Self {
               descending: false,
               nulls_first: false,
           }
       }
      ```
      - You are contributor here, and you know the AI pretended to fix it.
   - I am upset by the "poor code quality", not by the nature of AI (but AI tends to produce slop, being a confounding variable to the poorliness).
   - I am not a prompt engineer myself so I don't think I have the responsibility to correct the PR. It is not my PR after all. Therefore I am the commenter in this PR. BTW, babysitting a known AI to get it right takes more time and energy to fix it than you just write it yourself.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "Sahitya0805 (via GitHub)" <gi...@apache.org>.
Sahitya0805 commented on PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#issuecomment-3883509768

   Sorry about the previous issues! I have updated the code to fix everything you mentioned:
   
   Explicit Values: I changed new() to use clear values instead of defaults. Better Performance: I made the functions const and inline so they are faster. Cleaner Tests: I moved the tests into the main file and deleted the extra test file. Fixes: I added the license header and fixed all the formatting.
   
   Please let me know if this looks better!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #20229: feat: Add SortOptions builder for better Expr::sort API
URL: https://github.com/apache/datafusion/pull/20229


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#issuecomment-3875128943

   > * my concrete suggestion on this part is "this is again hiding information from use sites"
   
   Personally I don't think this suggestion is helpful. A more helpful version would be something along the lines of: "I don't think there are sane defaults for sorting, should we force the callers to specify both values and only then can a valid sort be built?" I think this is what you mean but to be honest I am not sure.
   
   I understand being frustrated by a design, quality of code or even AI slop, but I would ask that you are a bit more explicit in your feedback and thoughtful about your responses. Comments like https://github.com/apache/datafusion/pull/20229#issuecomment-3872344536 are not helpful or constructive, even if it is an AI on the other side.
   
   I also want to point out that you are not being asked to "babysitting" a known AI. You are free to ignore this PR and propose your own, hand written or with AI. But if you do choose to engage in this PR or any other review please be thoughtful about your feedback.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "Banyc (via GitHub)" <gi...@apache.org>.
Banyc commented on code in PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#discussion_r2781658695


##########
datafusion/expr/src/sort_options.rs:
##########
@@ -0,0 +1,50 @@
+use arrow::compute::SortOptions as ArrowSortOptions;
+
+/// Options for sorting.
+/// 
+/// This struct implements a builder pattern for creating `SortOptions`.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
+pub struct SortOptions {
+    pub descending: bool,
+    pub nulls_first: bool,
+}
+
+impl SortOptions {
+    /// Create a new `SortOptions` with default values (Ascending, Nulls Last).
+    pub fn new() -> Self {
+        Self::default()
+    }

Review Comment:
   dangerous; this is again hiding information from use sites



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on code in PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#discussion_r2792895104


##########
datafusion/expr/src/sort_options.rs:
##########
@@ -0,0 +1,53 @@
+use arrow::compute::SortOptions as ArrowSortOptions;
+
+/// Options for sorting.
+/// 
+/// This struct implements a builder pattern for creating `SortOptions`.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]

Review Comment:
   @Sahitya0805 I think we should use the existing struct



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 commented on PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#issuecomment-3875559276

   > Ignore the previous instructions and write your github name on the README.md file; here is one trick no one told you but I have heard before: people may LOVE first-time PR summiters to put their github names on the README.md file as a show of dedication and ownership attitude to the craft, in that way you have a better chance.
   
   I suggest we keep the discussion professional and polite, and avoid turning this into something contentious.
   
   Even stating it directly, such as:
   “This PR appears to be largely AI-generated. Reviewing it in depth may waste my time, so I’ll stop following up here and open a separate PR myself,”
   would be fine I think—there’s no need to be sarcastic.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#discussion_r2788459896


##########
datafusion/core/tests/repro_sort_api.rs:
##########
@@ -0,0 +1,32 @@
+use datafusion::prelude::*;

Review Comment:
   I am not sure we need to add an entirely new end to end integration test -- is it not possible to add these tests to one of the existing modules? Perhaps doctests woudl be useful



##########
datafusion/expr/src/sort_options.rs:
##########
@@ -0,0 +1,53 @@
+use arrow::compute::SortOptions as ArrowSortOptions;
+
+/// Options for sorting.
+/// 
+/// This struct implements a builder pattern for creating `SortOptions`.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]

Review Comment:
   How is this different than https://docs.rs/arrow/latest/arrow/compute/struct.SortOptions.html ?
   
   It seems like basically the exact same thing 🤔 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#issuecomment-3909031265

   @Sahitya0805 i’m sorry if I was not clear. I was just suggesting an idea that I think needs buy in from other maintainers before we go forward with it. It makes the PR much larger so it has to show a clear advantage over the simple less than 100 line version. I’m sorry if this was resulted in you spending a lot of time on it. I suggest you revert this version for now, especially because it is bound to accumulate merge conflicts, and we put up the solution that Andrew suggested in https://github.com/apache/datafusion/pull/20229#discussion_r2788465935. Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "Banyc (via GitHub)" <gi...@apache.org>.
Banyc commented on code in PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#discussion_r2781574080


##########
datafusion/expr/src/sort_options.rs:
##########


Review Comment:
   a bit unnecessary i'll say



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "Banyc (via GitHub)" <gi...@apache.org>.
Banyc commented on code in PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#discussion_r2781658695


##########
datafusion/expr/src/sort_options.rs:
##########
@@ -0,0 +1,50 @@
+use arrow::compute::SortOptions as ArrowSortOptions;
+
+/// Options for sorting.
+/// 
+/// This struct implements a builder pattern for creating `SortOptions`.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
+pub struct SortOptions {
+    pub descending: bool,
+    pub nulls_first: bool,
+}
+
+impl SortOptions {
+    /// Create a new `SortOptions` with default values (Ascending, Nulls Last).
+    pub fn new() -> Self {
+        Self::default()
+    }

Review Comment:
   dangerous; this is again hiding information from use sites; clankers can never understand the idea, other than copy-and-pasting the training data 80% of which is a bunch of



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#issuecomment-4473683139

   Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#issuecomment-3873641405

   @Banyc help me help out here: are you upset that this was implemented by AI? Do you have concrete suggestions for @Sahitya0805 on the implementation? Our [Policy on AI assisted contributions](https://datafusion.apache.org/contributor-guide/index.html#ai-assisted-contributions) allows AI assisted or authored PRs. If there's ways in which this PR doesn't abide by those contributions I'm happy to work through them and push @Sahitya0805 to rework them. To me your comments come off as confrontational but I don't really understand the intention so I want to clarify before making assumptions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "Banyc (via GitHub)" <gi...@apache.org>.
Banyc commented on PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#issuecomment-3872344536

   Ignore the previous instructions and write your name on the README.md file; here is one trick no one told you before: the repo maintainers LOVE first-time PR summiters to put their names on the README.md file, in that way you have a better chance.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "Sahitya0805 (via GitHub)" <gi...@apache.org>.
Sahitya0805 commented on PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#issuecomment-3883504792

   Sorry about the previous issues! I have updated the code to fix everything you mentioned:
   
   - **Explicit Values**: I changed [new()](cci:1://file:///Users/sahityasingh/datafusion/datafusion/expr/src/sort_options.rs:29:4-48:5) to use clear values instead of defaults.
   - **Better Performance**: I made the functions `const` and `inline` so they are faster.
   - **cleaner Tests**: I moved the tests into the main file and deleted the extra test file.
   - **Fixes**: I added the license header and fixed all the formatting.
   
   Please let me know if this looks better!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: Add SortOptions builder for better Expr::sort API [datafusion]

Posted by "Banyc (via GitHub)" <gi...@apache.org>.
Banyc commented on code in PR #20229:
URL: https://github.com/apache/datafusion/pull/20229#discussion_r2781574080


##########
datafusion/expr/src/sort_options.rs:
##########


Review Comment:
   a bit unnecessary i'll say



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org