[GitHub] [arrow] erezrokah opened a new issue, #35948: [Go] Marshaling `int64` or `unit64` arrays to JSON can cause loss of data

["erezrokah (via GitHub)" <gi...@apache.org>]

erezrokah opened a new issue, #35948:
URL: https://github.com/apache/arrow/issues/35948

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   When marshaling `array.Int64` or `array.Uint64` to JSON the original `int64` or `unit64` values can be lost.
   The cause is that the values are casted to `float64` before marshaling, and that type is not big enough to hold all possible `int64` or `unit64` values.
   
   Relevant code:
   https://github.com/apache/arrow/blob/4ec231b23432e9af340ed1a10d914f9bee84aca7/go/arrow/array/numeric.gen.go#L104
   https://github.com/apache/arrow/blob/4ec231b23432e9af340ed1a10d914f9bee84aca7/go/arrow/array/numeric.gen.go#L199
   
   Adding the following tests to [`numeric_test.go`](https://github.com/apache/arrow/blob/4ec231b23432e9af340ed1a10d914f9bee84aca7/go/arrow/array/numeric_test.go) result in failures:
   
   ```go
   func TestInt64MarshalJSON(t *testing.T) {
   	pool := memory.NewCheckedAllocator(memory.NewGoAllocator())
   	defer pool.AssertSize(t, 0)
   
   	var (
   		vs = []int64{-5474557666971701248}
   	)
   
   	b := array.NewInt64Builder(pool)
   	defer b.Release()
   
   	for _, v := range vs {
   		b.Append(v)
   	}
   
   	arr := b.NewArray().(*array.Int64)
   	defer arr.Release()
   
   	jsonBytes, err := json.Marshal(arr)
   	if err != nil {
   		t.Fatal(err)
   	}
   	got := string(jsonBytes)
   	want := `[-5474557666971701248]`
   	if got != want {
   		t.Fatalf("got=%s, want=%s", got, want)
   	}
   }
   
   func TestUInt64MarshalJSON(t *testing.T) {
   	pool := memory.NewCheckedAllocator(memory.NewGoAllocator())
   	defer pool.AssertSize(t, 0)
   
   	var (
   		vs = []uint64{14697929703826477056}
   	)
   
   	b := array.NewUint64Builder(pool)
   	defer b.Release()
   
   	for _, v := range vs {
   		b.Append(v)
   	}
   
   	arr := b.NewArray().(*array.Uint64)
   	defer arr.Release()
   
   	jsonBytes, err := json.Marshal(arr)
   	if err != nil {
   		t.Fatal(err)
   	}
   	got := string(jsonBytes)
   	want := `[14697929703826477056]`
   	if got != want {
   		t.Fatalf("got=%s, want=%s", got, want)
   	}
   }
   ```
   
   A possible solution is to only do the casting to `float64` when the array is of `unit8` like the comment says, or skip the casting for `int64` and `unit64` arrays
   
   ### Component(s)
   
   Go


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] erezrokah commented on issue #35948: [Go] Marshaling `int64` or `unit64` arrays to JSON can cause loss of data

["erezrokah (via GitHub)" <gi...@apache.org>]

erezrokah commented on issue #35948:
URL: https://github.com/apache/arrow/issues/35948#issuecomment-1579001731

   take


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade closed issue #35948: [Go] Marshaling `int64` or `unit64` arrays to JSON can cause loss of data

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade closed issue #35948: [Go] Marshaling `int64` or `unit64` arrays to JSON can cause loss of data 
URL: https://github.com/apache/arrow/issues/35948


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org