You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/04/24 12:26:40 UTC
[GitHub] [arrow-datafusion] alamb opened a new issue, #2326: Properly support arrow `Struct` types / Composite type in DataFusion
alamb opened a new issue, #2326:
URL: https://github.com/apache/arrow-datafusion/issues/2326
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
This ticket is designed to capture the work needed to properly support Arrow `Struct` types in DataFusion
https://arrow.apache.org/datafusion/user-guide/sql/sql_status.html says that nested types are not supported; The are not fully supported, but there are parts of the support already present such as a way to serialize them via ArrowWriter and using `field["nested_field"]` syntax
**Describe the solution you'd like**
Research, and describe / implement what is else remains for proper support.
Known issues so far:
- [ ] https://github.com/apache/arrow-datafusion/issues/2179 from @Cheappie
- [ ] https://github.com/apache/arrow-datafusion/issues/2043 from @lquerel
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alexwilcoxson-rel commented on issue #2326: [EPIC] Improved support for nested / structured types (`Struct` , `List`, `ListArray`, and other Composite types)
Posted by "alexwilcoxson-rel (via GitHub)" <gi...@apache.org>.
alexwilcoxson-rel commented on issue #2326:
URL: https://github.com/apache/arrow-datafusion/issues/2326#issuecomment-1699948540
Potential to add to list #7012
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [EPIC] Improved support for nested / structured types (`Struct` , `List`, `ListArray`, and other Composite types) [arrow-datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #2326:
URL: https://github.com/apache/arrow-datafusion/issues/2326#issuecomment-2022689299
We are starting to make progress on struct support --
There is a PR up to support `named_struct` https://github.com/apache/arrow-datafusion/pull/9743 and work afoot to support nicer literal syntax: https://github.com/apache/arrow-datafusion/issues/9820 🚀
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [EPIC] Improved support for nested / structured types (`Struct` , `List`, `ListArray`, and other Composite types) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #2326:
URL: https://github.com/apache/datafusion/issues/2326#issuecomment-2080448944
> Hi, i think unnest support for struct can be an item in this epic right?
That would make sense to me -- is there a ticket that describes what this means? I
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #2326: Properly support arrow `Struct` types / Composite type in DataFusion
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #2326:
URL: https://github.com/apache/arrow-datafusion/issues/2326#issuecomment-1152215337
Thanks for the investigation @nl5887 -- that sounds definitely plausible. Feel free to file a PR with proposed changed -- we would love to review them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] nl5887 commented on issue #2326: Properly support arrow `Struct` types / Composite type in DataFusion
Posted by GitBox <gi...@apache.org>.
nl5887 commented on issue #2326:
URL: https://github.com/apache/arrow-datafusion/issues/2326#issuecomment-1151582810
This https://github.com/apache/arrow-datafusion/blob/master/datafusion/core/src/physical_plan/file_format/mod.rs#L238 is one reason of errors related to column projection. It compares the complete enum, failing on different filed order.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] nl5887 commented on issue #2326: Properly support arrow `Struct` types / Composite type in DataFusion
Posted by GitBox <gi...@apache.org>.
nl5887 commented on issue #2326:
URL: https://github.com/apache/arrow-datafusion/issues/2326#issuecomment-1166540897
This one is also related: https://github.com/apache/arrow-datafusion/issues/2581
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [EPIC] Improved support for nested / structured types (`Struct` , `List`, `ListArray`, and other Composite types) [datafusion]
Posted by "duongcongtoai (via GitHub)" <gi...@apache.org>.
duongcongtoai commented on issue #2326:
URL: https://github.com/apache/datafusion/issues/2326#issuecomment-2080458071
i created a ticket: https://github.com/apache/datafusion/issues/10264
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
[GitHub] [arrow-datafusion] tv42 commented on issue #2326: Properly support arrow `Struct` types / Composite type in DataFusion
Posted by "tv42 (via GitHub)" <gi...@apache.org>.
tv42 commented on issue #2326:
URL: https://github.com/apache/arrow-datafusion/issues/2326#issuecomment-1440921697
Reminder to write docs: #1222
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [EPIC] Improved support for nested / structured types (`Struct` , `List`, `ListArray`, and other Composite types) [datafusion]
Posted by "toaiduongdh (via GitHub)" <gi...@apache.org>.
toaiduongdh commented on issue #2326:
URL: https://github.com/apache/datafusion/issues/2326#issuecomment-2080361039
Hi, i think unnest support for struct can be an item in this epic right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [EPIC] Improved support for nested / structured types (`Struct` , `List`, `ListArray`, and other Composite types) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #2326:
URL: https://github.com/apache/datafusion/issues/2326#issuecomment-2085321547
> i created a ticket: #10264
Thank you. I added this to the list in the ticket description
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org