You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "js8544 (via GitHub)" <gi...@apache.org> on 2023/09/14 06:19:50 UTC
[GitHub] [arrow] js8544 commented on issue #37678: [C++][Compute] Checked arithmetic functions are slow-ish
js8544 commented on issue #37678:
URL: https://github.com/apache/arrow/issues/37678#issuecomment-1718829195
`Add` doesn't check nulls and simply computes all pairs, null or not, while `AddChecked` only computes the valid pairs. (Code is [here](https://github.com/apache/arrow/blob/main/cpp/src/arrow/compute/kernels/scalar_arithmetic.cc#L1323), it uses MakeArithmeticFunction**NotNull** while `Add` uses MakeArithmeticFunction). I guess it was assumed that `__builtin_*_overflow` is an expensive operation and should be avoided as much as possible. However, I ran some benchmarks and the results don't agree with this assumption.
With null checking as it currently is, `AddChecked` is 15\~40x slower and the benefits of null checks only happen when >95% values are null, which I'd say is a rare case.
Without null checking, i.e. also using `MakeArithmeticFunction` for `AddChecked`, it is only 2.5 times slower, no matter the null proportion of the input:
This explains the difference between @pitrou and @cyb70289's results.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org