You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alexandreyc (via GitHub)" <gi...@apache.org> on 2023/05/11 19:07:40 UTC
[GitHub] [arrow-rs] alexandreyc opened a new pull request, #4205: Add RecordBatchWriter trait and implement it for CSV, JSON, IPC and Parquet
alexandreyc opened a new pull request, #4205:
URL: https://github.com/apache/arrow-rs/pull/4205
# Which issue does this PR close?
This PR doesn't close any particular issue.
# Rationale for this change
I found myself needing to work generically with writers of record batches and I need a common interface for doing that.
Do you find this useful? Feel free to reject the PR if you don't see any use case for it.
# What changes are included in this PR?
A new trait is introduced and implemented for CSV, JSON, IPC and Parquet :
```rust
/// Trait for types that can write `RecordBatch`'s.
pub trait RecordBatchWriter {
/// Write a single batch to the writer.
fn write(&mut self, batch: &RecordBatch) -> Result<(), ArrowError>;
}
```
# Are there any user-facing changes?
According to my analyses there are at least two breaking changes:
1. To use `write` clients now need to import `RecordBatchWriter`
2. `parquet::arrow_writer::ArrowWriter` now returns an `ArrowError` instead of `ParquetError`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on pull request #4205: Add RecordBatchWriter trait and implement it for CSV, JSON, IPC and Parquet
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on PR #4205:
URL: https://github.com/apache/arrow-rs/pull/4205#issuecomment-1544548358
Perhaps we could avoid this being a breaking change by not removing the member functions? This would also make the PR smaller? This sort of method overloading has well-defined semantics, which we exploit for things like Array::slice
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alexandreyc commented on pull request #4205: Add RecordBatchWriter trait and implement it for CSV, JSON, IPC and Parquet
Posted by "alexandreyc (via GitHub)" <gi...@apache.org>.
alexandreyc commented on PR #4205:
URL: https://github.com/apache/arrow-rs/pull/4205#issuecomment-1544554597
Oh yes you're right, I didn't this was possible... I will open a new PR. Thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alexandreyc closed pull request #4205: Add RecordBatchWriter trait and implement it for CSV, JSON, IPC and Parquet
Posted by "alexandreyc (via GitHub)" <gi...@apache.org>.
alexandreyc closed pull request #4205: Add RecordBatchWriter trait and implement it for CSV, JSON, IPC and Parquet
URL: https://github.com/apache/arrow-rs/pull/4205
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org