You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/01/13 09:52:55 UTC
[GitHub] [arrow] guyuqi opened a new pull request #12138: ARROW-15320: [Go] Implement memset_neon with Arm64 GoLang Assembly
guyuqi opened a new pull request #12138:
URL: https://github.com/apache/arrow/pull/12138
### Benchmark:
Before:
```
BenchmarkSet_32-46 50375077 20.79 ns/op 1539.46 MB/s
BenchmarkSet_64-46 33885868 35.44 ns/op 1806.12 MB/s
BenchmarkSet_500-46 4830316 244.8 ns/op 2042.77 MB/s
BenchmarkSet_512-46 4715967 250.0 ns/op 2047.66 MB/s
BenchmarkSet_1000-46 2502268 473.5 ns/op 2112.03 MB/s
BenchmarkSet_1024-46 2455328 484.5 ns/op 2113.36 MB/s
BenchmarkSet_2000-46 1284357 932.9 ns/op 2143.94 MB/s
BenchmarkSet_2048-46 1254939 952.7 ns/op 2149.64 MB/s
BenchmarkSet_4000-46 649353 1846 ns/op 2166.42 MB/s
BenchmarkSet_4096-46 632190 1892 ns/op 2164.46 MB/s
BenchmarkSet_8000-46 277814 3682 ns/op 2172.91 MB/s
BenchmarkSet_8192-46 281846 3774 ns/op 2170.57 MB/s
```
After:
```
BenchmarkSet_32-46 124619421 9.609 ns/op 3330.07 MB/s
BenchmarkSet_64-46 108461996 11.05 ns/op 5793.86 MB/s
BenchmarkSet_500-46 30755696 34.08 ns/op 14671.42 MB/s
BenchmarkSet_512-46 38423780 26.72 ns/op 19164.37 MB/s
BenchmarkSet_1000-46 25968873 43.92 ns/op 22770.37 MB/s
BenchmarkSet_1024-46 25674890 41.33 ns/op 24775.83 MB/s
BenchmarkSet_2000-46 15052582 75.31 ns/op 26557.55 MB/s
BenchmarkSet_2048-46 15718210 70.64 ns/op 28994.00 MB/s
BenchmarkSet_4000-46 8946339 130.3 ns/op 30700.07 MB/s
BenchmarkSet_4096-46 8686082 133.1 ns/op 30764.04 MB/s
BenchmarkSet_8000-46 4817458 245.3 ns/op 32614.89 MB/s
BenchmarkSet_8192-46 4715134 250.6 ns/op 32686.20 MB/s
```
Get 3x ~ 15x performance uplift.
### Test result: Passed
ARM_ENABLE_EXT=NEON make test
```
go test ./...
ok github.com/apache/arrow/go/v7/arrow (cached)
ok github.com/apache/arrow/go/v7/arrow/array (cached)
ok github.com/apache/arrow/go/v7/arrow/arrio (cached)
ok github.com/apache/arrow/go/v7/arrow/bitutil (cached)
? github.com/apache/arrow/go/v7/arrow/cdata [no test files]
ok github.com/apache/arrow/go/v7/arrow/csv (cached)
ok github.com/apache/arrow/go/v7/arrow/decimal128 (cached)
? github.com/apache/arrow/go/v7/arrow/endian [no test files]
ok github.com/apache/arrow/go/v7/arrow/flight (cached)
ok github.com/apache/arrow/go/v7/arrow/float16 (cached)
? github.com/apache/arrow/go/v7/arrow/internal/arrdata [no test files]
ok github.com/apache/arrow/go/v7/arrow/internal/arrjson (cached)
ok github.com/apache/arrow/go/v7/arrow/internal/cpu (cached)
? github.com/apache/arrow/go/v7/arrow/internal/debug [no test files]
? github.com/apache/arrow/go/v7/arrow/internal/flatbuf [no test files]
? github.com/apache/arrow/go/v7/arrow/internal/flight_integration [no test files]
? github.com/apache/arrow/go/v7/arrow/internal/flight_integration/cmd/arrow-flight-integration-client [no test files]
? github.com/apache/arrow/go/v7/arrow/internal/flight_integration/cmd/arrow-flight-integration-server [no test files]
? github.com/apache/arrow/go/v7/arrow/internal/testing/gen [no test files]
ok github.com/apache/arrow/go/v7/arrow/internal/testing/tools (cached)
? github.com/apache/arrow/go/v7/arrow/internal/testing/types [no test files]
ok github.com/apache/arrow/go/v7/arrow/ipc (cached)
ok github.com/apache/arrow/go/v7/arrow/ipc/cmd/arrow-cat (cached)
ok github.com/apache/arrow/go/v7/arrow/ipc/cmd/arrow-file-to-stream (cached)
ok github.com/apache/arrow/go/v7/arrow/ipc/cmd/arrow-json-integration-test (cached)
ok github.com/apache/arrow/go/v7/arrow/ipc/cmd/arrow-ls (cached)
ok github.com/apache/arrow/go/v7/arrow/ipc/cmd/arrow-stream-to-file (cached)
ok github.com/apache/arrow/go/v7/arrow/math (cached)
ok github.com/apache/arrow/go/v7/arrow/memory (cached)
ok github.com/apache/arrow/go/v7/arrow/scalar (cached)
ok github.com/apache/arrow/go/v7/arrow/tensor (cached)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12138: ARROW-15320: [Go] Implement memset_neon with Arm64 GoLang Assembly
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12138:
URL: https://github.com/apache/arrow/pull/12138#issuecomment-1012617957
Benchmark runs are scheduled for baseline = 03219e21b42f17294fba3b3d2b22a9117fe0f080 and contender = d19280898a1aceb3d7840368afc93ee3ed1a1ed2. d19280898a1aceb3d7840368afc93ee3ed1a1ed2 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/5dc9ab9b07334fab9bec4927d75f07cb...e628fa64fc154ed7a8cc7eefa6a6597c/)
[Failed :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/01349efd70ee49588a118f7ee1a70778...f9957e4e116f47da82412f4377b32378/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/8862bb268f5f446991f6f447dd96e91b...4798206d428a433c836d8029e76e92fa/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] zeroshade commented on pull request #12138: ARROW-15320: [Go] Implement memset_neon with Arm64 GoLang Assembly
Posted by GitBox <gi...@apache.org>.
zeroshade commented on pull request #12138:
URL: https://github.com/apache/arrow/pull/12138#issuecomment-1012611255
Thanks much! @guyuqi this looks great
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12138: ARROW-15320: [Go] Implement memset_neon with Arm64 GoLang Assembly
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12138:
URL: https://github.com/apache/arrow/pull/12138#issuecomment-1012617957
Benchmark runs are scheduled for baseline = 03219e21b42f17294fba3b3d2b22a9117fe0f080 and contender = d19280898a1aceb3d7840368afc93ee3ed1a1ed2. d19280898a1aceb3d7840368afc93ee3ed1a1ed2 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/5dc9ab9b07334fab9bec4927d75f07cb...e628fa64fc154ed7a8cc7eefa6a6597c/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/01349efd70ee49588a118f7ee1a70778...f9957e4e116f47da82412f4377b32378/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/8862bb268f5f446991f6f447dd96e91b...4798206d428a433c836d8029e76e92fa/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] zeroshade commented on pull request #12138: ARROW-15320: [Go] Implement memset_neon with Arm64 GoLang Assembly
Posted by GitBox <gi...@apache.org>.
zeroshade commented on pull request #12138:
URL: https://github.com/apache/arrow/pull/12138#issuecomment-1013214393
@guyuqi Awesome, i'll keep an eye out for the PRs, please make sure to tag me when you create them :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12138: ARROW-15320: [Go] Implement memset_neon with Arm64 GoLang Assembly
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12138:
URL: https://github.com/apache/arrow/pull/12138#issuecomment-1012617957
Benchmark runs are scheduled for baseline = 03219e21b42f17294fba3b3d2b22a9117fe0f080 and contender = d19280898a1aceb3d7840368afc93ee3ed1a1ed2. d19280898a1aceb3d7840368afc93ee3ed1a1ed2 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/5dc9ab9b07334fab9bec4927d75f07cb...e628fa64fc154ed7a8cc7eefa6a6597c/)
[Failed :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/01349efd70ee49588a118f7ee1a70778...f9957e4e116f47da82412f4377b32378/)
[Finished :arrow_down:0.22% :arrow_up:0.04%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/8862bb268f5f446991f6f447dd96e91b...4798206d428a433c836d8029e76e92fa/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #12138: ARROW-15320: [Go] Implement memset_neon with Arm64 GoLang Assembly
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #12138:
URL: https://github.com/apache/arrow/pull/12138#issuecomment-1011971740
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] guyuqi commented on pull request #12138: ARROW-15320: [Go] Implement memset_neon with Arm64 GoLang Assembly
Posted by GitBox <gi...@apache.org>.
guyuqi commented on pull request #12138:
URL: https://github.com/apache/arrow/pull/12138#issuecomment-1012673313
> Thanks much! @guyuqi this looks great. I don't have easy access to a NEON ARM64 architecture machine, do you think you could the same for the assembly files in https://github.com/apache/arrow/tree/master/go/parquet/internal/utils for the Parquet encoding / bit packing / etc. ?
Absolutely yes. :-)
I'd like to add the simlilar Arm64 Neon implementation in `go/parquet/internal/utils` for Parquet encoding / bit packing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #12138: ARROW-15320: [Go] Implement memset_neon with Arm64 GoLang Assembly
Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #12138:
URL: https://github.com/apache/arrow/pull/12138#issuecomment-1012617957
Benchmark runs are scheduled for baseline = 03219e21b42f17294fba3b3d2b22a9117fe0f080 and contender = d19280898a1aceb3d7840368afc93ee3ed1a1ed2. d19280898a1aceb3d7840368afc93ee3ed1a1ed2 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Scheduled] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/5dc9ab9b07334fab9bec4927d75f07cb...e628fa64fc154ed7a8cc7eefa6a6597c/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/01349efd70ee49588a118f7ee1a70778...f9957e4e116f47da82412f4377b32378/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/8862bb268f5f446991f6f447dd96e91b...4798206d428a433c836d8029e76e92fa/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] zeroshade edited a comment on pull request #12138: ARROW-15320: [Go] Implement memset_neon with Arm64 GoLang Assembly
Posted by GitBox <gi...@apache.org>.
zeroshade edited a comment on pull request #12138:
URL: https://github.com/apache/arrow/pull/12138#issuecomment-1012611255
Thanks much! @guyuqi this looks great. I don't have easy access to a NEON ARM64 architecture machine, do you think you could the same for the assembly files in https://github.com/apache/arrow/tree/master/go/parquet/internal/utils for the Parquet encoding / bit packing / etc. ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] zeroshade closed pull request #12138: ARROW-15320: [Go] Implement memset_neon with Arm64 GoLang Assembly
Posted by GitBox <gi...@apache.org>.
zeroshade closed pull request #12138:
URL: https://github.com/apache/arrow/pull/12138
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org