[jira] [Comment Edited] (ARROW-16716) [Benchmarks] Create Projection benchmark for Acero

["Ivan Chau (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17546060#comment-17546060 ] 

Ivan Chau edited comment on ARROW-16716 at 6/3/22 3:53 PM:
-----------------------------------------------------------

Hi [~westonpace], it's nice to meet you!

I've got a first draft of the benchmarks and have some clarification questions for you!

I am working on pushing the code soon, but the current benchmark right now is processing 1e6 elements with varying batch size. The pipeline includes a source node, a projection node which contains a field_ref and an expression (we carry over the same expressions used in ExecuteScalarExpressionOverhead and vary them per run), and then a basic sink (no agg or ordering).

I am currently using rows_per_second to measure performance, and I have found that they are not exactly the same as the ExecuteScalarExpressionOverhead, although the relative performance (comparing simple_expression vs complex_expression for a 2x speedup, for example), is very similar. This other overhead could be from the other `field_ref` operation I have in the projection, or the source/sink portions (I am not sure how hefty these are).

Here are some of my follow-ups as I tune these benchmarks:
 * How do we isolate and find the performance of _only_ the project node rather than source/project/sink
 * How to measure the data rate, since it seems we are measuring in rows / batches.
 * I saw some examples of controlling the threads used in ExecuteScalarExpressionOverhead, how can we control threads per core?

Thank you! I will have the code up shortly.

[^out]

^[^out_expression]^


was (Author: JIRAUSER290345):
Hi [~westonpace], it's nice to meet you!

I've got a first draft of the benchmarks and have some clarification questions for you!

I am working on pushing the code soon, but the current benchmark right now is processing 1e6 elements with varying batch size. The pipeline includes a source node, a projection node which contains a field_ref and an expression (we carry over the same expressions used in ExecuteScalarExpressionOverhead and vary them per run), and then a basic sink (no agg).

I am currently using rows_per_second to measure performance, and I have found that they are not exactly the same as the ExecuteScalarExpressionOverhead, although the relative performance (comparing simple_expression vs complex_expression for a 2x speedup, for example), is very similar. This other overhead could be from the other `field_ref` operation I have in the projection, or the source/sink portions (I am not sure how hefty these are).

Here are some of my follow-ups as I tune these benchmarks:
 * How do we isolate and find the performance of _only_ the project node rather than source/project/sink
 * How to measure the data rate, since it seems we are measuring in rows / batches.
 * I saw some examples of controlling the threads used in ExecuteScalarExpressionOverhead, how can we control threads per core?

Thank you! I will have the code up shortly.

[^out]

^[^out_expression]^

> [Benchmarks] Create Projection benchmark for Acero
> --------------------------------------------------
>
>                 Key: ARROW-16716
>                 URL: https://issues.apache.org/jira/browse/ARROW-16716
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: Benchmarking
>            Reporter: Li Jin
>            Priority: Major
>         Attachments: out, out_expression
>
>




--
This message was sent by Atlassian Jira
(v8.20.7#820007)