[GitHub] [arrow] pitrou commented on pull request #12755: ARROW-16014: [C++] Create more benchmarks for measuring expression evaluation overhead

[GitBox <gi...@apache.org>]

pitrou commented on PR #12755:
URL: https://github.com/apache/arrow/pull/12755#issuecomment-1098157510

   I think the benchmark parameters need a bit more care, I see long benchmark runtimes and very small numbers of iterations here:
   ```
   ExecuteScalarExpressionOverhead/complex_expression/rows_per_batch:10/threads:1          2984705514 ns   2983966976 ns            1 rows_per_second=3.35124M/s
   ExecuteScalarExpressionOverhead/complex_expression/rows_per_batch:10/threads:24          974849792 ns   23299109717 ns           24 rows_per_second=429.201k/s
   ExecuteScalarExpressionOverhead/complex_expression/rows_per_batch:100/threads:1          309637037 ns    309564292 ns            2 rows_per_second=32.3035M/s
   ExecuteScalarExpressionOverhead/complex_expression/rows_per_batch:100/threads:24          98364502 ns   2350758927 ns           24 rows_per_second=4.25395M/s
   ExecuteScalarExpressionOverhead/complex_expression/rows_per_batch:1000/threads:1          36930541 ns     36922101 ns           19 rows_per_second=270.84M/s
   ExecuteScalarExpressionOverhead/complex_expression/rows_per_batch:1000/threads:24         10100718 ns    240673758 ns           24 rows_per_second=41.55M/s
   ExecuteScalarExpressionOverhead/complex_expression/rows_per_batch:10000/threads:1          9879723 ns      9877473 ns           69 rows_per_second=1012.4M/s
   ExecuteScalarExpressionOverhead/complex_expression/rows_per_batch:10000/threads:24         1229958 ns     27766086 ns           24 rows_per_second=360.152M/s
   ExecuteScalarExpressionOverhead/complex_expression/rows_per_batch:100000/threads:1         7225455 ns      7223980 ns           97 rows_per_second=1.38428G/s
   ExecuteScalarExpressionOverhead/complex_expression/rows_per_batch:100000/threads:24         602946 ns     14455377 ns           48 rows_per_second=691.784M/s
   ExecuteScalarExpressionOverhead/complex_expression/rows_per_batch:1000000/threads:1        6897269 ns      6895650 ns          101 rows_per_second=1.45019G/s
   ExecuteScalarExpressionOverhead/complex_expression/rows_per_batch:1000000/threads:24       5401460 ns    128381172 ns           24 rows_per_second=77.893M/s
   ExecuteScalarExpressionOverhead/complex_expression/rows_per_batch:10000000/threads:1      12383094 ns     12375142 ns           57 rows_per_second=808.072M/s
   ExecuteScalarExpressionOverhead/complex_expression/rows_per_batch:10000000/threads:24      5386334 ns    125937873 ns           24 rows_per_second=79.4042M/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org