Re: [I] [C++][Python] Adding data to tdigest in pyarrow [arrow]

["thobai (via GitHub)" <gi...@apache.org>]

thobai commented on issue #35508:
URL: https://github.com/apache/arrow/issues/35508#issuecomment-2500034788

   @kat-grayson and @kinow: With which solution did you end up? I'm working on a distributed Polars project, which has similar requirements, but they're slightly different. We need to be able produce the tdigests on different workers (map) and then merge them on our head (reduce). We've implemented a Polars plugin called [polars-tdigest](https://github.com/appian/polars-tdigest) for this that wraps around the [tdigest Rust crate](https://github.com/MnO2/t-digest). Yet, the tdigest rust implementation isn't actively maintained and we're having issues with its accuracy, so we are looking for better maintained alternatives. Would be very interested which solution you picked


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org