You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "carloea2 (via GitHub)" <gi...@apache.org> on 2025/10/27 20:51:55 UTC
[D] disallowing sorting on binary type [texera]
GitHub user carloea2 created a discussion: disallowing sorting on binary type
### Task Summary
Add a frontend message explaining sorting on binary attributes is not supported and mention brief options such as extracting sortable features from them beforehand.
### Priority
P1 – Low
### Task Type
- [ ] Code Implementation
- [ ] Documentation
- [ ] Refactor / Cleanup
- [ ] Testing / QA
- [ ] DevOps / Deployment
GitHub link: https://github.com/apache/texera/discussions/4007
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
GitHub user Yicong-Huang edited a comment on the discussion: disallowing sorting on binary type
The previous issue's description is to "disallow sort on binary type". It will be misleading if we keep the same title and description. If we update it, it will be confusing (the discussion context will be wrong). Also this is a great case where an issue (clear todo) should be converted to discussion (unsure debate).
So I think it's better to convert the original issue to discussion, and for the new task, we should create a new task issue, instead of updating the old issue.
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14799769
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
GitHub user carloea2 added a comment to the discussion: disallowing sorting on binary type
You can't say they have nothing to do with sorting if in the end they affect the sorting, right?
I think we will not reach an agreement, if @chenlica agrees with you then I will close this issue and add binary sorting for the scala version.
@Yicong-Huang also agrees with you.
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797554
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
GitHub user carloea2 added a comment to the discussion: disallowing sorting on binary type
Yes, I think that is also unexpected for some users, and it may cause confusion for real users, example;
An user has a CSV with a column containing scientific notation numbers mixed with normal floats, if in my python code I sort with pandas default, then pandas will cast automatically to float, and the sort will be right.
In texera, now, when reading my CSV, then sorting by that column, the sort will be as strings resulting in a totally different output as in my python code, and since there is no hint in the UI that the column is string the user will have a bad experience.
//// Extra question
In Python, bytes are sequences of unsigned integers 0..255, and binary sequences compare lexicographically by those numeric byte values.
On the JVM (Java/Scala), byte/Byte is signed 8-bit two’s-complement (-128..127).
Is not this another source of problems when sorting bytes? I know similar things can apply to strings but managing bytes is more complex which means for users needing bytes sorting we should provide more Params on how to cast and manage the bytes, right?
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797552
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "aicam (via GitHub)" <gi...@apache.org>.
GitHub user aicam added a comment to the discussion: disallowing sorting on binary type
My understanding is both of the perspectives are right:
- @carloea2 is correct because sorting binary and bytes are meaningless and user may end up using the operator in wrong way
- @aglinxinyuan is correct because we should not intervene in user experience and give them full flexibility
my suggestion is when user select an attribute that is binary, we show them a small warning like this:
``` The selected attribute is binary type and will be sorted as bytes ```
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797555
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
GitHub user carloea2 added a comment to the discussion: disallowing sorting on binary type
In the PR of https://github.com/apache/texera/pull/3774 we were talking about sorting on binary attrs. The decision was based on the fact that the order may be different for every user, what do you think?
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797546
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan added a comment to the discussion: disallowing sorting on binary type
All the potential issues you described have nothing to do with sorting — they’re more about how to propagate the type correctly before sorting it. The sort operator should be general and capable of processing all data types. How to make the result correct is a separate issue. I don't think we should disable a function because we can't or we don't want to spend time making the result correct. Sort on binary type is supported in pretty much all the database systems, I think we should support it. I don’t have much insight into the technical difficulty since I didn’t review the PR, but as a user, I would like to sort the binary.
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797553
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan added a comment to the discussion: disallowing sorting on binary type
I do have the need to sort binary. For example, I can have number 1, 2, 3, and I want to sort them as binary. We don't need to limit the possibility of use cases.
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797547
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
GitHub user carloea2 added a comment to the discussion: disallowing sorting on binary type
What do you mean with sorting 1,2,3 as binary? are not they sorted as long or other types?
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797548
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: disallowing sorting on binary type
Make sure to mention related issue(s) and PR(s).
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-15411458
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan added a comment to the discussion: disallowing sorting on binary type
See the following workflow:
<img height="400" alt="Image" src="https://github.com/user-attachments/assets/da66ed3f-37fd-4b7b-a303-ee6d15fd1be9" />
<img height="400" alt="Image" src="https://github.com/user-attachments/assets/afd8091e-a20e-4279-8325-a819586941ad" />
Binary is simply the type used to decode the bytes. For example, if I have the characters '1', '2', and '3', and decode them using the binary type, I get the bytes 31, 32, and 33 in hex. However, I should still be able to sort them and later decode them as strings if needed.
My point is that the type used to decode the data should not affect the ability to sort the data, since we might choose a different decoding method later, or I just want to sort bytes.
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797549
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
GitHub user Yicong-Huang added a comment to the discussion: disallowing sorting on binary type
I converted this issue to a discussion. Seems we all agreed to add support for sorting binary. that's good. @carloea2 please create another task issue to track it.
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797573
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
GitHub user carloea2 added a comment to the discussion: disallowing sorting on binary type
Knime allows binary sorting:
<img width="1919" height="1020" alt="Image" src="https://github.com/user-attachments/assets/fbfb2c84-fc74-457d-91ca-3ae829a0621d" />
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797557
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
GitHub user carloea2 added a comment to the discussion: disallowing sorting on binary type
After reviewing other platforms, I agree we should enable sorting on binary attributes, KNIME supports it, and it should be a small change on our side.
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797561
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
GitHub user carloea2 added a comment to the discussion: disallowing sorting on binary type
Ok, thank you.
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14799628
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan added a comment to the discussion: disallowing sorting on binary type
> I want to confirm if the next behavior is what an avg user will expect, if I input 1, 2, 10 then the sort result is 1, 10, 2, is this desired for the avg user, will this not lead to confusion or even errors if the user do not notice they are using binary type?
>
> <img width="1666" height="905" alt="Image" src="https://github.com/user-attachments/assets/b674dfa7-ee77-41a5-af59-b855b22b7608" />
User should know they are using binary type. We don't want to drop the support of something because user might make mistake. The same claim is also true for int and string. For example, what if user input 1, 2, 10 as string? Is the sorting result 1,10,2 unexpected?
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797551
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
GitHub user carloea2 added a comment to the discussion: disallowing sorting on binary type
I want to confirm if the next behavior is what an avg user will expect, if I input 1, 2, 10 then the sort result is 1, 10, 2, is this desired for the avg user, will this not lead to confusion or even errors if the user do not notice they are using binary type?
<img width="1666" height="905" alt="Image" src="https://github.com/user-attachments/assets/b674dfa7-ee77-41a5-af59-b855b22b7608" />
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797550
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
GitHub user carloea2 added a comment to the discussion: disallowing sorting on binary type
Hello good night,
I think RapidMiner does not have binary array data type: https://docs.rapidminer.com/9.8/studio/getting-started/important-terms.html#data-types
Also once the tables are loaded they do not restrict the sort based on any data type.
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797559
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
GitHub user carloea2 added a comment to the discussion: disallowing sorting on binary type
Yes this is done
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-15411517
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
GitHub user carloea2 closed a discussion: disallowing sorting on binary type
### Task Summary
Add a frontend message explaining sorting on binary attributes is not supported and mention brief options such as extracting sortable features from them beforehand.
### Priority
P1 – Low
### Task Type
- [ ] Code Implementation
- [ ] Documentation
- [ ] Refactor / Cleanup
- [ ] Testing / QA
- [ ] DevOps / Deployment
GitHub link: https://github.com/apache/texera/discussions/4007
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan added a comment to the discussion: disallowing sorting on binary type
Why do we disallow sort for binary?
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797545
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: disallowing sorting on binary type
How about RapidMiner?
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797558
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: disallowing sorting on binary type
Thanks for the clarification. I agree.
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14799872
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
GitHub user Yicong-Huang edited a comment on the discussion: disallowing sorting on binary type
The previous issue's description is to "disallow sort on binary type". And now we agree to support it instead. It will be misleading if we keep the same title and description. If we update it, it will be confusing (the discussion context will be wrong). Also this is a great case where an issue (clear todo) should be converted to discussion (unsure debate).
So I think it's better to convert the original issue to discussion, and for the new task, we should create a new task issue, instead of updating the old issue.
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14799769
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "carloea2 (via GitHub)" <gi...@apache.org>.
GitHub user carloea2 added a comment to the discussion: disallowing sorting on binary type
The conclusion was to allow binary sort
The issue is:
https://github.com/apache/texera/issues/4013
The PR is:
https://github.com/apache/texera/pull/4014
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-15411533
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: disallowing sorting on binary type
In this case, I suggest we support sorting on binary attributes. @aglinxinyuan @aicam @carloea2 please share your thoughts so that we can finish this issue.
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797560
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "aicam (via GitHub)" <gi...@apache.org>.
GitHub user aicam added a comment to the discussion: disallowing sorting on binary type
> In this case, I suggest we support sorting on binary attributes. [@aglinxinyuan](https://github.com/aglinxinyuan) [@aicam](https://github.com/aicam) [@carloea2](https://github.com/carloea2) please share your thoughts so that we can finish this issue.
I agree
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797562
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: disallowing sorting on binary type
Each perspective has its own merits. To help us decide a design, can we evaluate existing systems and see their design decision?
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14797556
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: disallowing sorting on binary type
@Yicong-Huang Just curious, why cannot we use the previous issue? Why did we convert it to a discussion?
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14799702
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
GitHub user Yicong-Huang added a comment to the discussion: disallowing sorting on binary type
The previous issue's description is to "disallow sort". It will be misleading if we keep the same title and description. If we update it, it will be confusing (the discussion context will be wrong). Also this is a great case where an issue (clear todo) should be converted to discussion (unsure debate).
So I think it's better to convert the original issue to discussion, and for the new task, we should create a new task issue, instead of updating the old issue.
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-14799769
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org
Re: [D] disallowing sorting on binary type [texera]
Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: disallowing sorting on binary type
@carloea2 Are we done with this discussion? If so, we can close it.
GitHub link: https://github.com/apache/texera/discussions/4007#discussioncomment-15411454
----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org