You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Moelf (via GitHub)" <gi...@apache.org> on 2023/04/13 04:24:07 UTC
[GitHub] [arrow-julia] Moelf opened a new issue, #427: `Tuple{UInt8}` not getting correctly written out
Moelf opened a new issue, #427:
URL: https://github.com/apache/arrow-julia/issues/427
similar to #411 but corresponds to `fixedsizelist.jl`:
```julia
julia> data1 = (; x = [(0x01, 0x02), (0x03, 0x04)])
julia> Arrow.write("/tmp/julia1.feather", data1)
julia> data2 = (; x = [b"\x01\x02", b"\x03\x04"])
julia> Arrow.write("/tmp/julia2.feather", data2)
julia> data3 = (; x = [(0x0001, 0x0002), (0x0003, 0x0004)])
julia> Arrow.write("/tmp/julia3.feather", data3)
```
```python
In [12]: pyarrow.feather.read_table("/tmp/julia1.feather")["x"]
Out[12]:
<pyarrow.lib.ChunkedArray object at 0x7fd62050c400>
[
[
0102,
0304
]
]
In [13]: pyarrow.feather.read_table("/tmp/julia2.feather")["x"]
Out[13]:
<pyarrow.lib.ChunkedArray object at 0x7fd62387ee30>
[
[
0102,
0304
]
]
In [14]: pyarrow.feather.read_table("/tmp/julia3.feather")["x"]
Out[14]:
<pyarrow.lib.ChunkedArray object at 0x7fd62046da30>
[
[
[
1,
2
],
[
3,
4
]
]
]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `NTuple{UInt8}` not getting correctly written out [arrow-julia]
Posted by "Moelf (via GitHub)" <gi...@apache.org>.
Moelf commented on issue #427:
URL: https://github.com/apache/arrow-julia/issues/427#issuecomment-1922237337
fairly critical, bump?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-julia] quinnj commented on issue #427: `NTuple{UInt8}` not getting correctly written out
Posted by "quinnj (via GitHub)" <gi...@apache.org>.
quinnj commented on issue #427:
URL: https://github.com/apache/arrow-julia/issues/427#issuecomment-1577892969
Yeah, I agree this isn't ideal. At the time, I thought this was probably a reasonable way to translate to the arrow fixed size binary data type, but in reality, we should have tried a way to limit to only `Base.CodeUnits` like we do now for the list data type. The problem is that we now unequivocally treat `Base.CodeUnits` as list, so there's not a straightforward way to say, "hey, I have a vector of fixed size binary data and want the fixed size binary arrrow data type". We could create a wrapper like `Arrow.FixedSizeBinary` that people would have to use explicitly, but that's a bit annoying. Let me think on this one for just a bit.
In any case, we would probably want to modify the `FixedSizeListKind` in ArrowTypes to also have a 3rd type parameter to track whether the fixed size should be binary or not (since we don't want to unequivocally treat `UInt8` eltype as binary, which is the core issue here).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-julia] Moelf commented on issue #427: `NTuple{UInt8}` not getting correctly written out
Posted by "Moelf (via GitHub)" <gi...@apache.org>.
Moelf commented on issue #427:
URL: https://github.com/apache/arrow-julia/issues/427#issuecomment-1577903472
let me know if you want me to try my hands on this one (once you have a design idea)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org