You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/06/24 10:29:49 UTC
[GitHub] [arrow-rs] tustvold opened a new issue, #1938: Add coerce_types flag to parquet ArrowWriter
tustvold opened a new issue, #1938:
URL: https://github.com/apache/arrow-rs/issues/1938
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
As discussed in #1666 not all types can be represented within a parquet schema.
**Describe the solution you'd like**
The consensus appears to be to:
* By default faithfully round-trip the source data, performing no potentially lossy type conversion
* Add a coerce_types flag that will use the arrow cast kernels to coerce incompatible types prior to writing them
**Describe alternatives you've considered**
See #1666
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add coerce_types flag to parquet ArrowWriter [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #1938:
URL: https://github.com/apache/arrow-rs/issues/1938#issuecomment-2867783295
@dsgibbons would you be willing to make a PR to complete Interval and Timestamp so we can close this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add coerce_types flag to parquet ArrowWriter [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #1938:
URL: https://github.com/apache/arrow-rs/issues/1938#issuecomment-2850908980
I think this issue was done in https://github.com/apache/arrow-rs/pull/6840
If that is not correct, please reopen / let me know what else needs to be done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add coerce_types flag to parquet ArrowWriter [arrow-rs]
Posted by "dsgibbons (via GitHub)" <gi...@apache.org>.
dsgibbons commented on issue #1938:
URL: https://github.com/apache/arrow-rs/issues/1938#issuecomment-2294861820
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add coerce_types flag to parquet ArrowWriter [arrow-rs]
Posted by "dsgibbons (via GitHub)" <gi...@apache.org>.
dsgibbons commented on issue #1938:
URL: https://github.com/apache/arrow-rs/issues/1938#issuecomment-2868728359
@alamb yes, but will probably take a while (1-2 months). If someone else wants to finish this off in the meantime then go ahead. I'll comment once start working on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add coerce_types flag to parquet ArrowWriter [arrow-rs]
Posted by "nssalian (via GitHub)" <gi...@apache.org>.
nssalian commented on issue #1938:
URL: https://github.com/apache/arrow-rs/issues/1938#issuecomment-3084866185
@alamb, is this an active issue? I'm happy to take this on if no one else is working on it at the moment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add coerce_types flag to parquet ArrowWriter [arrow-rs]
Posted by "dsgibbons (via GitHub)" <gi...@apache.org>.
dsgibbons commented on issue #1938:
URL: https://github.com/apache/arrow-rs/issues/1938#issuecomment-3085645262
I have not done any work on this @nssalian, so feel free to take this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add coerce_types flag to parquet ArrowWriter [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #1938: Add coerce_types flag to parquet ArrowWriter
URL: https://github.com/apache/arrow-rs/issues/1938
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add coerce_types flag to parquet ArrowWriter [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #1938:
URL: https://github.com/apache/arrow-rs/issues/1938#issuecomment-2867782428
reopening per @dsgibbons 's comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add coerce_types flag to parquet ArrowWriter [arrow-rs]
Posted by "dsgibbons (via GitHub)" <gi...@apache.org>.
dsgibbons commented on issue #1938:
URL: https://github.com/apache/arrow-rs/issues/1938#issuecomment-2865890300
@alamb my original PR only handled Date64. I'm not sure if Interval and Timestamp are still outstanding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add coerce_types flag to parquet ArrowWriter [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #1938:
URL: https://github.com/apache/arrow-rs/issues/1938#issuecomment-2868750777
Thank you @dsgibbons !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add coerce_types flag to parquet ArrowWriter [arrow-rs]
Posted by "CuteChuanChuan (via GitHub)" <gi...@apache.org>.
CuteChuanChuan commented on issue #1938:
URL: https://github.com/apache/arrow-rs/issues/1938#issuecomment-3796847172
Hi @alamb , could I give it a try?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add coerce_types flag to parquet ArrowWriter [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #1938:
URL: https://github.com/apache/arrow-rs/issues/1938#issuecomment-3802061570
Please do -- thanks @CuteChuanChuan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add coerce_types flag to parquet ArrowWriter [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #1938:
URL: https://github.com/apache/arrow-rs/issues/1938#issuecomment-3084932985
I am not sure -- you will need to research if there is any remaining items to do
Thank you @nssalian 🙏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org