You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "yah01 (via GitHub)" <gi...@apache.org> on 2023/05/14 06:52:30 UTC

[GitHub] [arrow-rs] yah01 opened a new issue, #4214: Support to write FixedSizeList

yah01 opened a new issue, #4214:
URL: https://github.com/apache/arrow-rs/issues/4214

   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   Have array data which are all with fixed size, when I tried to write the `RecordBatch` with FixedSizeList column, it failed and reported:
   ```
   "NYI("Datatype FixedSizeList(Field { name: \"item\", data_type: Float32, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, 128) is not yet supported")"
   ```
   
   **Describe the solution you'd like**
   I have searched this error and found at:
   https://github.com/apache/arrow-rs/blob/0190408147a34c6c08fcc9ba57443c629c678ca6/parquet/src/arrow/arrow_writer/levels.rs#L161
   
   It looks like not hard to support `FixedSizeList` just like `List` (not sure), the only thing need to do is constructing the `LevelInfoBuilder` for `FixedSizeList`.
   
   **Describe alternatives you've considered**
   No
   
   **Additional context**
   I'm happy to take it if it's simple as I described above
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] tustvold closed issue #4214: Support writing FixedSizeList to Parquet

Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold closed issue #4214: Support writing FixedSizeList to Parquet
URL: https://github.com/apache/arrow-rs/issues/4214


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] dexterduck commented on issue #4214: Support writing FixedSizeList to Parquet

Posted by "dexterduck (via GitHub)" <gi...@apache.org>.
dexterduck commented on issue #4214:
URL: https://github.com/apache/arrow-rs/issues/4214#issuecomment-1557293905

   @tustvold I'd be happy to take over the implementation for this (assuming you're not already working on it). I believe I have a working proof of concept, I just need to integrate it into the codebase and add tests.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] tustvold commented on issue #4214: Support to write FixedSizeList

Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #4214:
URL: https://github.com/apache/arrow-rs/issues/4214#issuecomment-1547009167

   I think this should be relatively straightforward, as far as anything involving dremel is straightforward. This series of posts may be helpful for whoever picks this one up - https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] tustvold commented on issue #4214: Support writing FixedSizeList to Parquet

Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #4214:
URL: https://github.com/apache/arrow-rs/issues/4214#issuecomment-1557296962

   Always happy to review PRs, FWIW I think the hard part is the read side


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] tustvold commented on issue #4214: Support writing FixedSizeList to Parquet

Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #4214:
URL: https://github.com/apache/arrow-rs/issues/4214#issuecomment-1573883442

   `label_issue.py` automatically added labels {'parquet'} from #4226


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org