You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "anjakefala (via GitHub)" <gi...@apache.org> on 2024/07/12 21:57:38 UTC

[I] Extend support for BYTE_STREAM_SPLIT to FIXED_LEN_BYTE_ARRAY, INT32, and INT64 primitive types [arrow-rs]

anjakefala opened a new issue, #6048:
URL: https://github.com/apache/arrow-rs/issues/6048

   Please correct me if I'm wrong! It seems arrow-rs has added BYTE_STREAM_SPLIT support for [float types](https://github.com/apache/arrow-rs/pull/5293), but not for other numerical types like INT32. 
   
   Since then, the Parquet spec has been expanded to extend BYTE_STREAM_SPLIT encoding to other numerical primitive types: https://github.com/apache/parquet-format/pull/229. The C++ PoC is here: https://github.com/apache/arrow/pull/40094. 
   
   It would be good for arrow-rs to support BYTE_STREAM_SPLIT encoding additionally for FIXED_LEN_BYTE_ARRAY, INT32, and INT64. 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Extend support for BYTE_STREAM_SPLIT to FIXED_LEN_BYTE_ARRAY, INT32, and INT64 primitive types [arrow-rs]

Posted by "etseidl (via GitHub)" <gi...@apache.org>.
etseidl commented on issue #6048:
URL: https://github.com/apache/arrow-rs/issues/6048#issuecomment-2251352794

   @anjakefala are you working on this? If not I have some free cycles to devote to this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Extend support for BYTE_STREAM_SPLIT to FIXED_LEN_BYTE_ARRAY, INT32, and INT64 primitive types [arrow-rs]

Posted by "etseidl (via GitHub)" <gi...@apache.org>.
etseidl commented on issue #6048:
URL: https://github.com/apache/arrow-rs/issues/6048#issuecomment-2271906992

   @anjakefala please let me know if you have performance issues, particularly with FIXED_LEN_BYTE_ARRAY data. That was particularly problematic on the read side. 😦 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Extend support for BYTE_STREAM_SPLIT to FIXED_LEN_BYTE_ARRAY, INT32, and INT64 primitive types [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #6048: Extend support for BYTE_STREAM_SPLIT to FIXED_LEN_BYTE_ARRAY, INT32, and INT64 primitive types
URL: https://github.com/apache/arrow-rs/issues/6048


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Extend support for BYTE_STREAM_SPLIT to FIXED_LEN_BYTE_ARRAY, INT32, and INT64 primitive types [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #6048:
URL: https://github.com/apache/arrow-rs/issues/6048#issuecomment-2322898549

   `label_issue.py` automatically added labels {'parquet'} from #6159


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Extend support for BYTE_STREAM_SPLIT to FIXED_LEN_BYTE_ARRAY, INT32, and INT64 primitive types [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #6048:
URL: https://github.com/apache/arrow-rs/issues/6048#issuecomment-2271842377

   > Thank you so much @etseidl and @alamb! =)
   
   Also @mapleFU  🙏 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Extend support for BYTE_STREAM_SPLIT to FIXED_LEN_BYTE_ARRAY, INT32, and INT64 primitive types [arrow-rs]

Posted by "anjakefala (via GitHub)" <gi...@apache.org>.
anjakefala commented on issue #6048:
URL: https://github.com/apache/arrow-rs/issues/6048#issuecomment-2271807421

   Thank you so much @etseidl and @alamb! =)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Extend support for BYTE_STREAM_SPLIT to FIXED_LEN_BYTE_ARRAY, INT32, and INT64 primitive types [arrow-rs]

Posted by "anjakefala (via GitHub)" <gi...@apache.org>.
anjakefala commented on issue #6048:
URL: https://github.com/apache/arrow-rs/issues/6048#issuecomment-2252026272

   @etseidl I am not! I would love it if you picked it up. =)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org