You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/06/02 10:16:13 UTC
[GitHub] [arrow] jorisvandenbossche commented on pull request #10410: ARROW-10640: [C++] A, "if_else" ("where") kernel to combine two arrays based on a mask
jorisvandenbossche commented on pull request #10410:
URL: https://github.com/apache/arrow/pull/10410#issuecomment-852904327
I did a quick comparison with `np.where` (using plain float64 arrays without nulls to have it comparable), and it seems this implementation is already doing quite good compared to numpy (using a release build):
```
In [19]: N = 10_000_000
...: arr1 = np.random.randn(N)
...: arr2 = np.random.randn(N)
...: mask = np.random.randint(0, 2, N).astype(bool)
...:
...: pa_arr1 = pa.array(arr1)
...: pa_arr2 = pa.array(arr2)
...: pa_mask = pa.array(mask)
...:
In [20]: %timeit np.where(mask, arr1, arr2)
82.3 ms ± 8.76 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [21]: %timeit pc.if_else(pa_mask, pa_arr1, pa_arr2)
50.4 ms ± 6.08 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org