[jira] [Created] (ARROW-17896) [Go][Parquet] panic: fatal error: found bad pointer in Go heap on ByteArrayColumnChunkReader.Skip

["Ben (Jira)" <ji...@apache.org>]

Ben created ARROW-17896:
---------------------------

             Summary: [Go][Parquet] panic: fatal error: found bad pointer in Go heap on ByteArrayColumnChunkReader.Skip
                 Key: ARROW-17896
                 URL: https://issues.apache.org/jira/browse/ARROW-17896
             Project: Apache Arrow
          Issue Type: Bug
            Reporter: Ben


{code:java}
runtime: pointer 0xc001470001 to unused region of span span.base()=0xc000e78000 span.limit=0xc000e7a000 span.state=1
fatal error: found bad pointer in Go heap (incorrect use of unsafe or cgo?)

runtime stack:
runtime.throw({0x15ac9e9?, 0x3?})
	/usr/local/go/src/runtime/panic.go:992 +0x71 fp=0xc00010fee0 sp=0xc00010feb0 pc=0x438351
runtime.badPointer(0x7f16917f4d70, 0xc00010ff80?, 0x0, 0xc0006ef1e0?)
	/usr/local/go/src/runtime/mbitmap.go:368 +0x150 fp=0xc00010ff30 sp=0xc00010fee0 pc=0x414490
runtime.findObject(0xc000193a00?, 0xc0006ef1e0?, 0xc00010ffb0?)
	/usr/local/go/src/runtime/mbitmap.go:410 +0xa6 fp=0xc00010ff68 sp=0xc00010ff30 pc=0x414626
runtime.wbBufFlush1(0xc000066000)
	/usr/local/go/src/runtime/mwbbuf.go:260 +0xe5 fp=0xc00010ffb0 sp=0xc00010ff68 pc=0x433525
runtime.wbBufFlush.func1()
	/usr/local/go/src/runtime/mwbbuf.go:201 +0x25 fp=0xc00010ffc8 sp=0xc00010ffb0 pc=0x433365
runtime.systemstack()
	/usr/local/go/src/runtime/asm_amd64.s:469 +0x49 fp=0xc00010ffd0 sp=0xc00010ffc8 pc=0x468ea9

goroutine 27 [running]:
runtime.systemstack_switch()
	/usr/local/go/src/runtime/asm_amd64.s:436 fp=0xc0007790c0 sp=0xc0007790b8 pc=0x468e40
runtime.wbBufFlush(0x0?, 0x0?)
	/usr/local/go/src/runtime/mwbbuf.go:200 +0x6c fp=0xc0007790e0 sp=0xc0007790c0 pc=0x4333ec
runtime.wbBufFlush(0xc00091e190, 0xc0014870eb)
	<autogenerated>:1 +0x2a fp=0xc000779100 sp=0xc0007790e0 pc=0x46d3aa
runtime.gcWriteBarrier()
	/usr/local/go/src/runtime/asm_amd64.s:1669 +0xa3 fp=0xc000779180 sp=0xc000779100 pc=0x46b003
runtime.gcWriteBarrierDX()
	/usr/local/go/src/runtime/asm_amd64.s:1697 +0x7 fp=0xc000779188 sp=0xc000779180 pc=0x46b087
github.com/apache/arrow/go/parquet/internal/encoding.(*ByteArrayDictConverter).Copy(0xc00091ca80?, {0x1252500?, 0xc0006c6000?}, {0xc0005e7000?, 0x400?, 0x0?})
	/go/pkg/mod/github.com/apache/arrow/go/parquet@v0.0.0-20211112161151-bc219186db40/internal/encoding/typed_encoder.gen.go:1344 +0xa7 fp=0xc0007791b0 sp=0xc000779188 pc=0xbde387
github.com/apache/arrow/go/parquet/internal/utils.(*RleDecoder).GetBatchWithDictByteArray(0xc000276500, {0x1766bf8, 0xc000505140}, {0xc00091ca80, 0x229, 0x400})
	/go/pkg/mod/github.com/apache/arrow/go/parquet@v0.0.0-20211112161151-bc219186db40/internal/utils/typed_rle_dict.gen.go:1168 +0x1d0 fp=0xc000779238 sp=0xc0007791b0 pc=0xbac590
github.com/apache/arrow/go/parquet/internal/utils.(*RleDecoder).GetBatchWithDict(0x40a4cd?, {0x1766bf8?, 0xc000505140?}, {0x1252500?, 0xc0007792e0?})
	/go/pkg/mod/github.com/apache/arrow/go/parquet@v0.0.0-20211112161151-bc219186db40/internal/utils/rle.go:422 +0x1de fp=0xc000779290 sp=0xc000779238 pc=0xba4bbe
github.com/apache/arrow/go/parquet/internal/encoding.(*dictDecoder).decode(...)
	/go/pkg/mod/github.com/apache/arrow/go/parquet@v0.0.0-20211112161151-bc219186db40/internal/encoding/decoder.go:140
github.com/apache/arrow/go/parquet/internal/encoding.(*DictByteArrayDecoder).Decode(0xc00114b620?, {0xc00091ca80?, 0xc000276500?, 0x41fc10?})
	/go/pkg/mod/github.com/apache/arrow/go/parquet@v0.0.0-20211112161151-bc219186db40/internal/encoding/typed_encoder.gen.go:1250 +0x6d fp=0xc000779308 sp=0xc000779290 pc=0xbdd9cd
github.com/apache/arrow/go/parquet/file.(*ByteArrayColumnChunkReader).ReadBatch.func1(0x0, 0x229)
	/go/pkg/mod/github.com/apache/arrow/go/parquet@v0.0.0-20211112161151-bc219186db40/file/column_reader_types.gen.go:263 +0xa6 fp=0xc000779350 sp=0xc000779308 pc=0xbf0e86
github.com/apache/arrow/go/parquet/file.(*columnChunkReader).readBatch(0xc000e4aa00, 0x229, {0xc00091ca80, 0x3000, 0x3000}, {0xc00091ca80, 0x3000, 0x3000}, 0xc000779440)
	/go/pkg/mod/github.com/apache/arrow/go/parquet@v0.0.0-20211112161151-bc219186db40/file/column_reader.go:485 +0xcd fp=0xc0007793f8 sp=0xc000779350 pc=0xbeec4d
github.com/apache/arrow/go/parquet/file.(*ByteArrayColumnChunkReader).ReadBatch(0x0?, 0x7f16b85f9108?, {0xc00091ca80?, 0xc000580000?, 0x203000?}, {0xc00091ca80?, 0xc000779538?, 0x46b107?}, {0xc00091ca80, 0x3000, ...})
	/go/pkg/mod/github.com/apache/arrow/go/parquet@v0.0.0-20211112161151-bc219186db40/file/column_reader_types.gen.go:262 +0x87 fp=0xc000779478 sp=0xc0007793f8 pc=0xbf0d67
github.com/apache/arrow/go/parquet/file.(*ByteArrayColumnChunkReader).Skip.func1(0xc001054b40?, {0xc00091ca80?, 0x0?, 0x0?})
	/go/pkg/mod/github.com/apache/arrow/go/parquet@v0.0.0-20211112161151-bc219186db40/file/column_reader_types.gen.go:244 +0x1f8 fp=0xc000779570 sp=0xc000779478 pc=0xbf0c78
github.com/apache/arrow/go/parquet/file.(*columnChunkReader).skipValues(0xc000e4aa00, 0x629, 0xc000779608)
	/go/pkg/mod/github.com/apache/arrow/go/parquet@v0.0.0-20211112161151-bc219186db40/file/column_reader.go:442 +0x2a5 fp=0xc0007795f0 sp=0xc000779570 pc=0xbeeac5
github.com/apache/arrow/go/parquet/file.(*ByteArrayColumnChunkReader).Skip(0x10?, 0x176e738?)
	/go/pkg/mod/github.com/apache/arrow/go/parquet@v0.0.0-20211112161151-bc219186db40/file/column_reader_types.gen.go:242 +0x37 fp=0xc000779628 sp=0xc0007795f0 pc=0xbf0a57 {code}
I occasionally (but frequently) hit the above, +un-recoverable+ panic when parsing parquet files.

It does not look like this has anything to do with the parquet file itself, as repeatedly parsing the same file works out fine.

 

Using *github.com/apache/arrow/go/parquet v0.0.0-20211112161151-bc219186db40*

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)