You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "adriangb (via GitHub)" <gi...@apache.org> on 2026/10/18 13:41:15 UTC
[I] Make bloom filters / ArrowSbbf it's own crate [arrow-rs]
adriangb opened a new issue, #11681:
URL: https://github.com/apache/arrow-rs/issues/11681
I'm interested in using our bloom filter implementation in DataFusion outside of the context of parquet. To do that we'd want to have it be it's own crate. It seems like a good general implementation of bloom filters that is compatible w/ the arrow ecosystem. Shall we make it it's own crate so that it can be used outside of the context of parquet?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Make bloom filters / ArrowSbbf it's own crate [arrow-rs]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on issue #11681:
URL: https://github.com/apache/arrow-rs/issues/11681#issuecomment-6250610735
cc @Jefffrey @alamb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Make bloom filters / ArrowSbbf it's own crate [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #11681:
URL: https://github.com/apache/arrow-rs/issues/11681#issuecomment-6271872043
Seems like a reasonable idea to me. Maybe a crate like `parquet-bloom-filter` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Make bloom filters / ArrowSbbf it's own crate [arrow-rs]
Posted by "etseidl (via GitHub)" <gi...@apache.org>.
etseidl commented on issue #11681:
URL: https://github.com/apache/arrow-rs/issues/11681#issuecomment-6481611148
I've got #11975 (on top of #11953) submitted now that may simplify this some 🤷
PTAL 🙏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Make bloom filters / ArrowSbbf it's own crate [arrow-rs]
Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on issue #11681:
URL: https://github.com/apache/arrow-rs/issues/11681#issuecomment-6269740663
related issue https://github.com/apache/arrow-rs/issues/11518
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Make bloom filters / ArrowSbbf it's own crate [arrow-rs]
Posted by "Rich-T-kid (via GitHub)" <gi...@apache.org>.
Rich-T-kid commented on issue #11681:
URL: https://github.com/apache/arrow-rs/issues/11681#issuecomment-6276164493
this can be distinct from parquet, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Make bloom filters / ArrowSbbf it's own crate [arrow-rs]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on issue #11681:
URL: https://github.com/apache/arrow-rs/issues/11681#issuecomment-6276344150
That's what I'm not sure of: in implementation, yes certainly. In spirit: maybe not? If we are going to choose to match the encodings to the Parquet spec / implementations (so that these bloom filters and encodings can be used in parquet or interact w/ parquet's bloom filters) then having `parquet` in the name sounds reasonable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Make bloom filters / ArrowSbbf it's own crate [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #11681:
URL: https://github.com/apache/arrow-rs/issues/11681#issuecomment-6423214917
I think calling it `parquet-bloom-filter` would be appropriate as it implements the bloom filters specified in the Parquet spec; So even if you want to use it outside of parquet files, it would still be correct to label them "Parquet Bloom Filters" I think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Make bloom filters / ArrowSbbf it's own crate [arrow-rs]
Posted by "etseidl (via GitHub)" <gi...@apache.org>.
etseidl commented on issue #11681:
URL: https://github.com/apache/arrow-rs/issues/11681#issuecomment-6473273536
Late to the party, but this seems like a good idea to me too. I've been playing around with trying to do for Bloom filters what I've been doing for the page index (#11475 and prior), but I think I'll hold off for a bit and see where this work goes. Thanks @adriangb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Make bloom filters / ArrowSbbf it's own crate [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #11681:
URL: https://github.com/apache/arrow-rs/issues/11681#issuecomment-6481498640
I think this would probably be a pretty simple job for the AI machine to pull out into its own crate. We should then make sure the docs,etc are reasonable but othrwise let's try and do it for 60.2.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org