You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/01/13 22:56:46 UTC
[GitHub] [arrow] zeroshade opened a new pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
zeroshade opened a new pull request #12149:
URL: https://github.com/apache/arrow/pull/12149
This is the last large portion of code to get merged. anything after this is individual bug fixes or improvements. With the exception of features that aren't yet implemented in Go (like Dictionary Arrays and such) this brings the Go Parquet library pretty much to mostly feature parity with the C++ Parquet Library.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] zeroshade commented on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
zeroshade commented on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1026020029
@drauschenbach seeing as I don't currently have access to an arm64 Mac system, if you do manage to track down the cause of that crash (given the failure in the compress tests, I'm going to guess it is likely related to the github.com/klauspost/compress package I'm using), any information you find or a PR would be fantastic :smile:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] drauschenbach commented on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
drauschenbach commented on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1025990117
Also, in reference to the `PARQUET_TEST_DATA` env var mentioned above I found `ci/scripts/go_test.sh`, and that explains a required `-test` tag. But I also don't make it through that test suite. Many tests pass, until:
```
$ ci/scripts/go_test.sh `pwd`
...
~/workspaces/zeroshade-arrow
+ export PARQUET_TEST_DATA=/Users/davidr/workspaces/zeroshade-arrow/cpp/submodules/parquet-testing/data
+ PARQUET_TEST_DATA=/Users/davidr/workspaces/zeroshade-arrow/cpp/submodules/parquet-testing/data
+ pushd /Users/davidr/workspaces/zeroshade-arrow/go/parquet
~/workspaces/zeroshade-arrow/go/parquet ~/workspaces/zeroshade-arrow
++ go list ./...
++ grep -v vendor
+ for d in '$(go list ./... | grep -v vendor)'
+ go test -race -tags assert github.com/apache/arrow/go/v7/parquet
ok github.com/apache/arrow/go/v7/parquet 0.549s
+ for d in '$(go list ./... | grep -v vendor)'
+ go test -race -tags assert github.com/apache/arrow/go/v7/parquet/cmd/parquet_reader
? github.com/apache/arrow/go/v7/parquet/cmd/parquet_reader [no test files]
+ for d in '$(go list ./... | grep -v vendor)'
+ go test -race -tags assert github.com/apache/arrow/go/v7/parquet/cmd/parquet_schema
? github.com/apache/arrow/go/v7/parquet/cmd/parquet_schema [no test files]
+ for d in '$(go list ./... | grep -v vendor)'
+ go test -race -tags assert github.com/apache/arrow/go/v7/parquet/compress
SIGABRT: abort
PC=0x1c1f199b8 m=0 sigcode=0
goroutine 0 [idle]:
runtime: unknown pc 0x1c1f199b8
stack: frame={sp:0x16d793220, fp:0x0} stack=[0x16d714468,0x16d7934e0)
...
0x000000016d793300: 0x000000016d793320 0x4e070001c1d84c8c
0x000000016d793310: 0x00000000003c0000 0x000000016d793330
goroutine 1 [running, locked to thread]:
runtime.systemstack_switch()
/opt/homebrew/Cellar/go/1.17.3/libexec/src/runtime/asm_arm64.s:187 +0x8 fp=0xc00003e6b0 sp=0xc00003e6a0 pc=0x102712678
runtime.newm1(0xc000042d80)
/opt/homebrew/Cellar/go/1.17.3/libexec/src/runtime/proc.go:2246 +0xa8 fp=0xc00003e700 sp=0xc00003e6b0 pc=0x1026e5948
runtime.newm(0x102a79a70, 0x0, 0xffffffffffffffff)
/opt/homebrew/Cellar/go/1.17.3/libexec/src/runtime/proc.go:2230 +0x11c fp=0xc00003e740 sp=0xc00003e700 pc=0x1026e586c
runtime.startTemplateThread()
/opt/homebrew/Cellar/go/1.17.3/libexec/src/runtime/proc.go:2271 +0xec fp=0xc00003e770 sp=0xc00003e740 pc=0x1026e5abc
runtime.main()
/opt/homebrew/Cellar/go/1.17.3/libexec/src/runtime/proc.go:234 +0x204 fp=0xc00003e7d0 sp=0xc00003e770 pc=0x1026e1264
runtime.goexit()
/opt/homebrew/Cellar/go/1.17.3/libexec/src/runtime/asm_arm64.s:1133 +0x4 fp=0xc00003e7d0 sp=0xc00003e7d0 pc=0x102714af4
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] zeroshade commented on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
zeroshade commented on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1025992928
So, for the `PARQUET_TEST_DATA` it refers to the directory that pulls down the sample parquet files which are in the submodule loaded in the cpp source as you can see referenced by the value it is set to in that script.
As far as that crash you are seeing, that's interesting and not one I had encountered. Judging from the directory paths I assume you're running this on a Mac using an ARM64 CPU?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] drauschenbach commented on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
drauschenbach commented on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1026249180
> if you do manage to track down the cause of that crash (given the failure in the compress tests, I'm going to guess it is likely related to the github.com/klauspost/compress package I'm using), any information you find or a PR would be fantastic
I am re-running tests from the arrow master branch now (after your 2 merges) and am unable to reproduce any problems on Apple M1. So, maybe it was an intermittent non-deterministic event? I'll keep my eyes open.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1021770422
Benchmark runs are scheduled for baseline = 345609b61674cedc1daa3f1bf5d6b64266b19b26 and contender = c5f400461f6d2be836d30df4626fed4d59107015. c5f400461f6d2be836d30df4626fed4d59107015 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/77919f1fe9384f03b857f977a428e0dd...b6266e792e88462badd7a4bcea8bd43f/)
[Failed :arrow_down:1.27% :arrow_up:0.85%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/c604c4784e1147dc91533b54a002d5b4...a4ffc026754a469f821ccdfec52dddf8/)
[Finished :arrow_down:0.52% :arrow_up:0.04%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/cb3cee8911dc4de9b38960d52b0738ee...93e5a4880f23490595da95eb0c96e661/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1021770422
Benchmark runs are scheduled for baseline = 345609b61674cedc1daa3f1bf5d6b64266b19b26 and contender = c5f400461f6d2be836d30df4626fed4d59107015. c5f400461f6d2be836d30df4626fed4d59107015 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/77919f1fe9384f03b857f977a428e0dd...b6266e792e88462badd7a4bcea8bd43f/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/c604c4784e1147dc91533b54a002d5b4...a4ffc026754a469f821ccdfec52dddf8/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/cb3cee8911dc4de9b38960d52b0738ee...93e5a4880f23490595da95eb0c96e661/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] zeroshade commented on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
zeroshade commented on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1025988162
@drauschenbach So that's actually a bug I found also and put up a PR for. It came about due to a change I made in the interface to simplify things and then I forgot to update the mainprogs.
Please try using the branch in PR: https://github.com/apache/arrow/pull/12303 until it gets merged. Try that out and if it doesn't work let me know by commenting on that PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] drauschenbach commented on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
drauschenbach commented on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1025997702
> Please try using the branch in PR: #12303 until it gets merged. Try that out and if it doesn't work let me know by commenting on that PR
Thanks! I'll check it out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1021770422
Benchmark runs are scheduled for baseline = 345609b61674cedc1daa3f1bf5d6b64266b19b26 and contender = c5f400461f6d2be836d30df4626fed4d59107015. c5f400461f6d2be836d30df4626fed4d59107015 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Scheduled] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/77919f1fe9384f03b857f977a428e0dd...b6266e792e88462badd7a4bcea8bd43f/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/c604c4784e1147dc91533b54a002d5b4...a4ffc026754a469f821ccdfec52dddf8/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/cb3cee8911dc4de9b38960d52b0738ee...93e5a4880f23490595da95eb0c96e661/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] zeroshade closed pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
zeroshade closed pull request #12149:
URL: https://github.com/apache/arrow/pull/12149
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] drauschenbach commented on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
drauschenbach commented on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1025997231
> Judging from the directory paths I assume you're running this on a Mac using an ARM64 CPU?
Correct. Ah, I just noticed that Mac arm64 is not in the GitHub CI build matrix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] Ignalina commented on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
Ignalina commented on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1021778236
Lovly work !!!!! The pqarrow module is Very helpful for us beginners !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] drauschenbach edited a comment on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
drauschenbach edited a comment on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1025968885
I'm looking to get up to speed on the Go bindings through this PR, and so far I can get most of the tests to run (not including ones that require a `PARQUET_TEST_DATA` dir, which I'm not sure what to specify):
```sh
$ ci/scripts/go_build.sh `pwd`
$ cd go; go test -v ./...
(no failures)
```
But I'm struggling to kick the tires on the `parquet_reader` and `parquet_schema` commands. I've tried a few Parquet files and get the same result:
```sh
$ ~/go/bin/parquet_schema ~/nyc-taxi-data/2009/01/data.parquet
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x100689fec]
goroutine 1 [running]:
github.com/apache/arrow/go/v7/parquet/file.NewParquetReader({0x100750e00, 0x14000138240}, {0x140001b1f38, 0x2, 0x2})
/Users/davidr/workspaces/zeroshade-arrow/go/parquet/file/file_reader.go:144 +0x7c
github.com/apache/arrow/go/v7/parquet/file.OpenParquetFile({0x16f923984, 0x3f}, 0x0, {0x140001b1f38, 0x2, 0x2})
/Users/davidr/workspaces/zeroshade-arrow/go/parquet/file/file_reader.go:132 +0x110
main.main()
/Users/davidr/workspaces/zeroshade-arrow/go/parquet/cmd/parquet_schema/main.go:37 +0xb8
```
Any tips?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] drauschenbach edited a comment on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
drauschenbach edited a comment on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1025990117
Also, in reference to the `PARQUET_TEST_DATA` env var mentioned above I found `ci/scripts/go_test.sh`, and that explains a required `-test` tag. But I also don't make it through that test suite. Many tests pass, until:
```
$ git submodule init
$ git submodule update
$ ci/scripts/go_test.sh `pwd`
...
~/workspaces/zeroshade-arrow
+ export PARQUET_TEST_DATA=/Users/davidr/workspaces/zeroshade-arrow/cpp/submodules/parquet-testing/data
+ PARQUET_TEST_DATA=/Users/davidr/workspaces/zeroshade-arrow/cpp/submodules/parquet-testing/data
+ pushd /Users/davidr/workspaces/zeroshade-arrow/go/parquet
~/workspaces/zeroshade-arrow/go/parquet ~/workspaces/zeroshade-arrow
++ go list ./...
++ grep -v vendor
+ for d in '$(go list ./... | grep -v vendor)'
+ go test -race -tags assert github.com/apache/arrow/go/v7/parquet
ok github.com/apache/arrow/go/v7/parquet 0.549s
+ for d in '$(go list ./... | grep -v vendor)'
+ go test -race -tags assert github.com/apache/arrow/go/v7/parquet/cmd/parquet_reader
? github.com/apache/arrow/go/v7/parquet/cmd/parquet_reader [no test files]
+ for d in '$(go list ./... | grep -v vendor)'
+ go test -race -tags assert github.com/apache/arrow/go/v7/parquet/cmd/parquet_schema
? github.com/apache/arrow/go/v7/parquet/cmd/parquet_schema [no test files]
+ for d in '$(go list ./... | grep -v vendor)'
+ go test -race -tags assert github.com/apache/arrow/go/v7/parquet/compress
SIGABRT: abort
PC=0x1c1f199b8 m=0 sigcode=0
goroutine 0 [idle]:
runtime: unknown pc 0x1c1f199b8
stack: frame={sp:0x16d793220, fp:0x0} stack=[0x16d714468,0x16d7934e0)
...
0x000000016d793300: 0x000000016d793320 0x4e070001c1d84c8c
0x000000016d793310: 0x00000000003c0000 0x000000016d793330
goroutine 1 [running, locked to thread]:
runtime.systemstack_switch()
/opt/homebrew/Cellar/go/1.17.3/libexec/src/runtime/asm_arm64.s:187 +0x8 fp=0xc00003e6b0 sp=0xc00003e6a0 pc=0x102712678
runtime.newm1(0xc000042d80)
/opt/homebrew/Cellar/go/1.17.3/libexec/src/runtime/proc.go:2246 +0xa8 fp=0xc00003e700 sp=0xc00003e6b0 pc=0x1026e5948
runtime.newm(0x102a79a70, 0x0, 0xffffffffffffffff)
/opt/homebrew/Cellar/go/1.17.3/libexec/src/runtime/proc.go:2230 +0x11c fp=0xc00003e740 sp=0xc00003e700 pc=0x1026e586c
runtime.startTemplateThread()
/opt/homebrew/Cellar/go/1.17.3/libexec/src/runtime/proc.go:2271 +0xec fp=0xc00003e770 sp=0xc00003e740 pc=0x1026e5abc
runtime.main()
/opt/homebrew/Cellar/go/1.17.3/libexec/src/runtime/proc.go:234 +0x204 fp=0xc00003e7d0 sp=0xc00003e770 pc=0x1026e1264
runtime.goexit()
/opt/homebrew/Cellar/go/1.17.3/libexec/src/runtime/asm_arm64.s:1133 +0x4 fp=0xc00003e7d0 sp=0xc00003e7d0 pc=0x102714af4
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1012592131
https://issues.apache.org/jira/browse/ARROW-15331
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] drauschenbach commented on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
drauschenbach commented on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1025968885
I'm looking to get up to speed on the Go bindings through this PR, and so far I can get most of the tests to run (not including ones that require a `PARQUET_TEST_DATA` dir, which I'm not sure what to specify):
```sh
$ ci/scripts/go_build.sh `pwd`
$ cd go; go get -v ./...
(no failures)
```
But I'm struggling to kick the tires on the `parquet_reader` and `parquet_schema` commands. I've tried a few Parquet files and get the same result:
```sh
$ ~/go/bin/parquet_schema ~/nyc-taxi-data/2009/01/data.parquet
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x100689fec]
goroutine 1 [running]:
github.com/apache/arrow/go/v7/parquet/file.NewParquetReader({0x100750e00, 0x14000138240}, {0x140001b1f38, 0x2, 0x2})
/Users/davidr/workspaces/zeroshade-arrow/go/parquet/file/file_reader.go:144 +0x7c
github.com/apache/arrow/go/v7/parquet/file.OpenParquetFile({0x16f923984, 0x3f}, 0x0, {0x140001b1f38, 0x2, 0x2})
/Users/davidr/workspaces/zeroshade-arrow/go/parquet/file/file_reader.go:132 +0x110
main.main()
/Users/davidr/workspaces/zeroshade-arrow/go/parquet/cmd/parquet_schema/main.go:37 +0xb8
```
Any tips?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] zeroshade commented on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
zeroshade commented on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1013305477
Ok *finally* fixed the race condition that it found and this is ready for review whenever you have the opportunity @emkornfield
Thankfully this is the last major big merge for the parquet lib :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1021770422
Benchmark runs are scheduled for baseline = 345609b61674cedc1daa3f1bf5d6b64266b19b26 and contender = c5f400461f6d2be836d30df4626fed4d59107015. c5f400461f6d2be836d30df4626fed4d59107015 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/77919f1fe9384f03b857f977a428e0dd...b6266e792e88462badd7a4bcea8bd43f/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/c604c4784e1147dc91533b54a002d5b4...a4ffc026754a469f821ccdfec52dddf8/)
[Finished :arrow_down:0.52% :arrow_up:0.04%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/cb3cee8911dc4de9b38960d52b0738ee...93e5a4880f23490595da95eb0c96e661/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] zeroshade commented on pull request #12149: ARROW-15331: [Go][Parquet] Add pqarrow package for direct Parquet <--> Arrow conversion
Posted by GitBox <gi...@apache.org>.
zeroshade commented on pull request #12149:
URL: https://github.com/apache/arrow/pull/12149#issuecomment-1026001035
@drauschenbach Yea, the CI matrix runs on a mac using amd64, and we have Linux arm64v8 run by travis CI but we don't have MacOS + arm64
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org