You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Elena Henderson (Jira)" <ji...@apache.org> on 2022/10/27 23:05:00 UTC
[jira] [Comment Edited] (ARROW-18183) cpp-micro benchmarks are failing on mac arm machine
[ https://issues.apache.org/jira/browse/ARROW-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17625352#comment-17625352 ]
Elena Henderson edited comment on ARROW-18183 at 10/27/22 11:04 PM:
--------------------------------------------------------------------
cpp-micro benchmarks started failing on `test-mac-arm` since this change: [https://github.com/apache/arrow/commit/ec579df631deaa8f6186208ed2a4ebec00581dfa]
First failed build: [https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/1628]
Steps to reproduce:
1. Run benchmarks for commit [https://github.com/apache/arrow/commit/89c0214fa43f8d1bf2e19e3bae0fc3009df51e15], which was made right before
[https://github.com/apache/arrow/commit/ec579df631deaa8f6186208ed2a4ebec00581dfa]
{code:java}
cd ~
rm -rf arrow
git clone https://github.com/apache/arrow.git
cd arrow
git checkout 89c0214fa43f8d1bf2e19e3bae0fc3009df51e15
conda create -y -n arrow-commit -c conda-forge \
--file ci/conda_env_unix.txt \
--file ci/conda_env_cpp.txt \
--file ci/conda_env_python.txt \
compilers \
python=3.8 \
pandas
conda activate arrow-commit
pip install -r python/requirements-build.txt -r python/requirements-test.txt
source dev/conbench_envs/hooks.sh set_arrow_build_and_run_env_vars
export RANLIB=`which $RANLIB`
export AR=`which $AR`
export ARROW_JEMALLOC=OFF
pip install -e dev/archery
archery benchmark run
{code}
Note benchmarks are NOT failing!
2. Run benchmarks for [https://github.com/apache/arrow/commit/ec579df631deaa8f6186208ed2a4ebec00581dfa] when cpp-micro benchmarks started failing:
{code:java}
cd ~
rm -rf arrow
git clone https://github.com/apache/arrow.git
cd arrow
git checkout ec579df631deaa8f6186208ed2a4ebec00581dfa
conda create -y -n arrow-commit -c conda-forge \
--file ci/conda_env_unix.txt \
--file ci/conda_env_cpp.txt \
--file ci/conda_env_python.txt \
compilers \
python=3.8 \
pandas
conda activate arrow-commit
pip install -r python/requirements-build.txt -r python/requirements-test.txt
source dev/conbench_envs/hooks.sh set_arrow_build_and_run_env_vars
export RANLIB=`which $RANLIB`
export AR=`which $AR`
export ARROW_JEMALLOC=OFF
pip install -e dev/archery
archery benchmark run
{code}
Note benchmarks stop running:
{code:java}
...
CoalesceScalarStringBench/6 6174977 ns 6174982 ns 111 bytes_per_second=664.007M/s items_per_second=169.81M/s length=1048.58k null%=99 num_args=2
ChooseBench64/1048576/0 6787276 ns 6787272 ns 103 bytes_per_second=1.15105G/s items_per_second=154.492M/s
ChooseBench64/1048576/99 6722603 ns 6722606 ns 104 bytes_per_second=1.16201G/s items_per_second=155.963M/s
/Users/voltrondata/arrow/cpp/src/arrow/memory_pool.cc:113: Unsupported backend 'mimalloc' specified in ARROW_DEFAULT_MEMORY_POOL (supported backends are 'system')
Unable to determine clock rate from sysctl: hw.cpufrequency: No such file or directory
This does not affect benchmark measurements, only the metadata output.
2022-10-27T18:58:59-04:00
Running /var/folders/d9/n0lbzbf16nvgt7b83xhcssm80000gn/T/arrow-archery-18pv4wou/WORKSPACE/build/release/arrow-dataset-scanner-benchmark
Run on (8 X 24.2407 MHz CPU s)
CPU Caches:
L1 Data 64 KiB
L1 Instruction 128 KiB
L2 Unified 4096 KiB (x8)
Load Average: 15.79, 15.29, 10.64
---------------------------------------------------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations UserCounters...
---------------------------------------------------------------------------------------------------------------------------------------------------
MinimalEndToEndBench/num_batches:1000/batch_size:10/scan_alg:0/iterations:10/real_time 23472042 ns 5977400 ns 10 bytes_per_second=2.03151M/s items_per_second=42.6039k/s
Traceback (most recent call last):
File "/Users/voltrondata/miniconda3/envs/arrow-commit/bin/archery", line 33, in <module>
sys.exit(load_entry_point('archery', 'console_scripts', 'archery')())
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/site-packages/click/core.py", line 1130, in _call_
return self.main(*args, **kwargs)
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/voltrondata/arrow/dev/archery/archery/cli.py", line 515, in benchmark_run
json_out = json.dumps(runner_base, cls=JsonEncoder)
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/json/_init_.py", line 234, in dumps
return cls(
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/Users/voltrondata/arrow/dev/archery/archery/benchmark/codec.py", line 35, in default
return BenchmarkRunnerCodec.encode(o)
File "/Users/voltrondata/arrow/dev/archery/archery/benchmark/codec.py", line 79, in encode
return {"suites": [BenchmarkSuiteCodec.encode(s) for s in br.suites]}
File "/Users/voltrondata/arrow/dev/archery/archery/benchmark/codec.py", line 79, in <listcomp>
return {"suites": [BenchmarkSuiteCodec.encode(s) for s in br.suites]}
File "/Users/voltrondata/arrow/dev/archery/archery/benchmark/runner.py", line 176, in suites
suite = self.suite(suite_name, suite_bin)
File "/Users/voltrondata/arrow/dev/archery/archery/benchmark/runner.py", line 152, in suite
results = suite_cmd.results(repetitions=self.repetitions)
File "/Users/voltrondata/arrow/dev/archery/archery/benchmark/google.py", line 63, in results
self.run(*argv, check=True)
File "/Users/voltrondata/arrow/dev/archery/archery/utils/command.py", line 78, in run
return subprocess.run(invocation, **kwargs)
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/var/folders/d9/n0lbzbf16nvgt7b83xhcssm80000gn/T/arrow-archery-18pv4wou/WORKSPACE/build/release/arrow-dataset-scanner-benchmark', '--benchmark_repetitions=1', '--benchmark_out=/var/folders/d9/n0lbzbf16nvgt7b83xhcssm80000gn/T/tmpzwwjwb94', '--benchmark_out_format=json']' died with <Signals.SIGBUS: 10>.
{code}
was (Author: elena@ursacomputing.com):
cpp-micro benchmarks started failing on `test-mac-arm` since this change: [https://github.com/apache/arrow/commit/ec579df631deaa8f6186208ed2a4ebec00581dfa]
First failed build: [https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/1628]
1. Run benchmarks for commit https://github.com/apache/arrow/commit/89c0214fa43f8d1bf2e19e3bae0fc3009df51e15, which was made right before
https://github.com/apache/arrow/commit/ec579df631deaa8f6186208ed2a4ebec00581dfa
```
cd ~
rm -rf arrow
git clone https://github.com/apache/arrow.git
cd arrow
git checkout 89c0214fa43f8d1bf2e19e3bae0fc3009df51e15
conda create -y -n arrow-commit -c conda-forge \
--file ci/conda_env_unix.txt \
--file ci/conda_env_cpp.txt \
--file ci/conda_env_python.txt \
compilers \
python=3.8 \
pandas
conda activate arrow-commit
pip install -r python/requirements-build.txt -r python/requirements-test.txt
source dev/conbench_envs/hooks.sh set_arrow_build_and_run_env_vars
export RANLIB=`which $RANLIB`
export AR=`which $AR`
export ARROW_JEMALLOC=OFF
pip install -e dev/archery
archery benchmark run
```
Note benchmarks are NOT failing
2. Run benchmarks for https://github.com/apache/arrow/commit/ec579df631deaa8f6186208ed2a4ebec00581dfa when cpp-micro benchmarks started failing:
```
cd ~
rm -rf arrow
git clone https://github.com/apache/arrow.git
cd arrow
git checkout ec579df631deaa8f6186208ed2a4ebec00581dfa
conda create -y -n arrow-commit -c conda-forge \
--file ci/conda_env_unix.txt \
--file ci/conda_env_cpp.txt \
--file ci/conda_env_python.txt \
compilers \
python=3.8 \
pandas
conda activate arrow-commit
pip install -r python/requirements-build.txt -r python/requirements-test.txt
source dev/conbench_envs/hooks.sh set_arrow_build_and_run_env_vars
export RANLIB=`which $RANLIB`
export AR=`which $AR`
export ARROW_JEMALLOC=OFF
pip install -e dev/archery
archery benchmark run
```
Note benchmarks stop running:
```
CoalesceScalarStringBench/6 6174977 ns 6174982 ns 111 bytes_per_second=664.007M/s items_per_second=169.81M/s length=1048.58k null%=99 num_args=2
ChooseBench64/1048576/0 6787276 ns 6787272 ns 103 bytes_per_second=1.15105G/s items_per_second=154.492M/s
ChooseBench64/1048576/99 6722603 ns 6722606 ns 104 bytes_per_second=1.16201G/s items_per_second=155.963M/s
/Users/voltrondata/arrow/cpp/src/arrow/memory_pool.cc:113: Unsupported backend 'mimalloc' specified in ARROW_DEFAULT_MEMORY_POOL (supported backends are 'system')
Unable to determine clock rate from sysctl: hw.cpufrequency: No such file or directory
This does not affect benchmark measurements, only the metadata output.
2022-10-27T18:58:59-04:00
Running /var/folders/d9/n0lbzbf16nvgt7b83xhcssm80000gn/T/arrow-archery-18pv4wou/WORKSPACE/build/release/arrow-dataset-scanner-benchmark
Run on (8 X 24.2407 MHz CPU s)
CPU Caches:
L1 Data 64 KiB
L1 Instruction 128 KiB
L2 Unified 4096 KiB (x8)
Load Average: 15.79, 15.29, 10.64
---------------------------------------------------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations UserCounters...
---------------------------------------------------------------------------------------------------------------------------------------------------
MinimalEndToEndBench/num_batches:1000/batch_size:10/scan_alg:0/iterations:10/real_time 23472042 ns 5977400 ns 10 bytes_per_second=2.03151M/s items_per_second=42.6039k/s
Traceback (most recent call last):
File "/Users/voltrondata/miniconda3/envs/arrow-commit/bin/archery", line 33, in <module>
sys.exit(load_entry_point('archery', 'console_scripts', 'archery')())
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/voltrondata/arrow/dev/archery/archery/cli.py", line 515, in benchmark_run
json_out = json.dumps(runner_base, cls=JsonEncoder)
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/json/__init__.py", line 234, in dumps
return cls(
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/Users/voltrondata/arrow/dev/archery/archery/benchmark/codec.py", line 35, in default
return BenchmarkRunnerCodec.encode(o)
File "/Users/voltrondata/arrow/dev/archery/archery/benchmark/codec.py", line 79, in encode
return \{"suites": [BenchmarkSuiteCodec.encode(s) for s in br.suites]}
File "/Users/voltrondata/arrow/dev/archery/archery/benchmark/codec.py", line 79, in <listcomp>
return \{"suites": [BenchmarkSuiteCodec.encode(s) for s in br.suites]}
File "/Users/voltrondata/arrow/dev/archery/archery/benchmark/runner.py", line 176, in suites
suite = self.suite(suite_name, suite_bin)
File "/Users/voltrondata/arrow/dev/archery/archery/benchmark/runner.py", line 152, in suite
results = suite_cmd.results(repetitions=self.repetitions)
File "/Users/voltrondata/arrow/dev/archery/archery/benchmark/google.py", line 63, in results
self.run(*argv, check=True)
File "/Users/voltrondata/arrow/dev/archery/archery/utils/command.py", line 78, in run
return subprocess.run(invocation, **kwargs)
File "/Users/voltrondata/miniconda3/envs/arrow-commit/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/var/folders/d9/n0lbzbf16nvgt7b83xhcssm80000gn/T/arrow-archery-18pv4wou/WORKSPACE/build/release/arrow-dataset-scanner-benchmark', '--benchmark_repetitions=1', '--benchmark_out=/var/folders/d9/n0lbzbf16nvgt7b83xhcssm80000gn/T/tmpzwwjwb94', '--benchmark_out_format=json']' died with <Signals.SIGBUS: 10>.
````
> cpp-micro benchmarks are failing on mac arm machine
> ---------------------------------------------------
>
> Key: ARROW-18183
> URL: https://issues.apache.org/jira/browse/ARROW-18183
> Project: Apache Arrow
> Issue Type: Bug
> Components: Benchmarking, C++
> Reporter: Elena Henderson
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)