You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "starpact (via GitHub)" <gi...@apache.org> on 2026/08/17 10:17:15 UTC

[PR] feat(parquet): expose writer abort [opendal]

starpact opened a new pull request, #8102:
URL: https://github.com/apache/opendal/pull/8102

   # Which issue does this PR close?
   
   Closes #8101.
   
   # Rationale for this change
   
   `parquet_opendal::AsyncWriter` does not expose OpenDAL's explicit abort operation. Applications currently need a custom `AsyncFileWriter` adapter to clean up failed writes, including multipart uploads.
   
   # What changes are included in this PR?
   
   - Add `AsyncWriter::abort`, mapping the OpenDAL error to `ParquetError::External` consistently with `write` and `complete`.
   - Update the abort test to terminate the writer explicitly instead of dropping it.
   
   # Are there any user-facing changes?
   
   Yes. Applications can retrieve `AsyncWriter` through `AsyncArrowWriter::into_inner()` and explicitly abort the underlying OpenDAL write.
   
   # AI Usage Statement
   
   Pi with the OpenAI GPT-5.6 model was used to inspect the implementation, make the focused change, and draft tests and documentation.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(parquet): expose writer abort [opendal]

Posted by "dentiny (via GitHub)" <gi...@apache.org>.
dentiny commented on code in PR #8102:
URL: https://github.com/apache/opendal/pull/8102#discussion_r3838483712


##########
integrations/parquet/src/async_writer.rs:
##########
@@ -80,6 +80,11 @@ impl AsyncWriter {
     pub fn new(writer: Writer) -> Self {
         Self { inner: writer }
     }
+
+    /// Aborts the write and cleans up data written by the underlying [`Writer`].

Review Comment:
   I would suggest simply use the `abort` function doc here: https://github.com/apache/opendal/blob/04364ea32390796730099203a211357100bc517c/core/core/src/raw/oio/write/api.rs#L38-L39



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(parquet): expose writer abort [opendal]

Posted by "starpact (via GitHub)" <gi...@apache.org>.
starpact commented on code in PR #8102:
URL: https://github.com/apache/opendal/pull/8102#discussion_r3796639399


##########
integrations/parquet/src/async_writer.rs:
##########
@@ -80,6 +80,11 @@ impl AsyncWriter {
     pub fn new(writer: Writer) -> Self {
         Self { inner: writer }
     }
+
+    /// Aborts the write and cleans up data written by the underlying [`Writer`].

Review Comment:
   It's not to be part of any interface, but to be called directly via `parquet::arrow::AsyncArrowWriter::into_inner`. I can see your point, but this only exposes what Writer already has, whether abort is supported remains runtime contract.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(parquet): expose writer abort [opendal]

Posted by "dentiny (via GitHub)" <gi...@apache.org>.
dentiny commented on code in PR #8102:
URL: https://github.com/apache/opendal/pull/8102#discussion_r3795832988


##########
integrations/parquet/src/async_writer.rs:
##########
@@ -80,6 +80,11 @@ impl AsyncWriter {
     pub fn new(writer: Writer) -> Self {
         Self { inner: writer }
     }
+
+    /// Aborts the write and cleans up data written by the underlying [`Writer`].

Review Comment:
   I think our interface doesn't guarantee data cleanup?
   https://github.com/apache/opendal/blob/04364ea32390796730099203a211357100bc517c/core/core/src/raw/oio/write/api.rs#L38-L39
   
   And not every storage backend supports abort, for example
   https://github.com/apache/opendal/blob/04364ea32390796730099203a211357100bc517c/core/services/alluxio/src/writer.rs#L70-L75
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(parquet): expose writer abort [opendal]

Posted by "starpact (via GitHub)" <gi...@apache.org>.
starpact commented on code in PR #8102:
URL: https://github.com/apache/opendal/pull/8102#discussion_r3796639399


##########
integrations/parquet/src/async_writer.rs:
##########
@@ -80,6 +80,11 @@ impl AsyncWriter {
     pub fn new(writer: Writer) -> Self {
         Self { inner: writer }
     }
+
+    /// Aborts the write and cleans up data written by the underlying [`Writer`].

Review Comment:
   It's not to be part of any interface, but to be called directly via `parquet::arrow::AsyncArrowWriter::into_inner`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(parquet): expose writer abort [opendal]

Posted by "dentiny (via GitHub)" <gi...@apache.org>.
dentiny commented on code in PR #8102:
URL: https://github.com/apache/opendal/pull/8102#discussion_r3838482407


##########
integrations/parquet/src/async_writer.rs:
##########
@@ -80,6 +80,11 @@ impl AsyncWriter {
     pub fn new(writer: Writer) -> Self {
         Self { inner: writer }
     }
+
+    /// Aborts the write and cleans up data written by the underlying [`Writer`].

Review Comment:
   sure -- but your comment "cleans up data written by the underlying [`Writer`]." is not accurate



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(parquet): expose writer abort [opendal]

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo merged PR #8102:
URL: https://github.com/apache/opendal/pull/8102


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org