[jira] [Commented] (ARROW-17584) [Go] Unable to build with tinygo

["Tim Schaub (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-17584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599144#comment-17599144 ] 

Tim Schaub commented on ARROW-17584:
------------------------------------

[~zeroshade] - it looks like things should work with `unsafe.Slice`.  I found that tinygo also does not yet implement `map.LoadAndDelete` for `sync`, but I've proposed adding that here https://github.com/tinygo-org/tinygo/pull/3118

I started to put together a change that converts the casting functions to use `unsafe.Slice`, but I'm struggling with what the appropriate len/cap should be.

For example, I was assuming `bytesTOUint64` in arrow/bitutil/bitutil.go might look something like this:

{code:go}
func bytesToUint64(b []byte) []uint64 {
	h := (*reflect.SliceHeader)(unsafe.Pointer(&b))

	return unsafe.Slice((*uint64)(unsafe.Pointer(h.Data)), len(b)/uint64SizeBytes)
}
{code}

But I'm uncertain if that is the correct length/capacity.  If you have thoughts on the correct implementation, I would be happy to put together a pull request that changes this and the other casting functions.

> [Go] Unable to build with tinygo
> --------------------------------
>
>                 Key: ARROW-17584
>                 URL: https://issues.apache.org/jira/browse/ARROW-17584
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Go
>            Reporter: Tim Schaub
>            Priority: Major
>
> I was hoping to use TinyGo to build WASM binaries with Arrow.  TinyGo can generate builds that are [1% the size|https://tinygo.org/getting-started/overview/#:~:text=The%20only%20difference%20here%2C%20is,used%2C%20and%20the%20associated%20runtime.&text=In%20this%20case%20the%20Go,size%20(251k%20before%20stripping)!] of those generated with Go (significant for applications hosted on the web).
> Arrow's use of `reflect.SliceHeader` fields limits the portability of the code.  For example, the `Len` and `Cap` fields are assumed to be `int` here: https://github.com/apache/arrow/blob/go/v9.0.0/go/arrow/bitutil/bitutil.go#L158-L159
> Go's [reflect package warns|https://github.com/golang/go/blob/go1.19/src/reflect/value.go#L2675-L2685] that the SliceHeader "cannot be used safely or portably and its representation may change in a later release."
> Attempts to build a WASM binary using the github.com/apache/arrow/go/v10 module result in failures like this:
> {code}
> tinygo build -tags noasm -o test.wasm ./main.go
> {code}
> {code}             
> # github.com/apache/arrow/go/v10/arrow/bitutil
> ../../go/pkg/mod/github.com/apache/arrow/go/v10@v10.0.0-20220831082949-cf27001da088/arrow/bitutil/bitutil.go:158:10: invalid operation: h.Len / uint64SizeBytes (mismatched types uintptr and int)
> ../../go/pkg/mod/github.com/apache/arrow/go/v10@v10.0.0-20220831082949-cf27001da088/arrow/bitutil/bitutil.go:159:10: invalid operation: h.Cap / uint64SizeBytes (mismatched types uintptr and int)
> {code}
> This happens because TinyGo uses `uintptr` for the corresponding types: https://github.com/tinygo-org/tinygo/blob/v0.25.0/src/reflect/value.go#L773-L777
> This feels like an issue with TinyGo, and it has been ticketed there multiple times (see https://github.com/tinygo-org/tinygo/issues/1284).  They lean on the warnings in the Go sources that use of the SliceHeader fields makes code unportable and suggest changes to the libraries that do not heed this warning.
> I don't have a suggested fix or alternative for Arrow's use of SliceHeader fields, but I'm wondering if there would be willingness on the part of this package to make WASM builds work with TinyGo.  Perhaps the TinyGo authors could also offer suggested changes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)