[PR] [core] Fix orphan file and NPE when a single file writer fails to open [paimon]

["wombatu-kun (via GitHub)" <gi...@apache.org>]

wombatu-kun opened a new pull request, #8923:
URL: https://github.com/apache/paimon/pull/8923

   ### Purpose
   
   `SingleFileWriter` and `FormatTableSingleFileWriter` open the output stream in their constructor, so the file exists on the filesystem before the format writer is built. The constructor sets `deleteFileUponAbort = true` up front, with the comment `// true first to clean file in exception`, but the cleanup only ran from `catch (IOException e)`, and only when `out != null`. Two problems follow.
   
   A non-IOException failure leaves an orphan file that no snapshot references, plus an unclosed stream. This is reachable: `file.compression` is not validated, so an unknown value reaches `CompressionKind.valueOf(compression.toUpperCase())` in `OrcWriterFactory` and throws `IllegalArgumentException`, and building a Parquet or ORC writer throws `UnsupportedOperationException("Unsupported type: ...")` for a type the format does not support.
   
   The cleanup also called the overridable `abort()` from the base constructor. `RowDataFileWriter` overrides it and iterates `auxiliaryFileWriters`, a field assigned after `super(...)`, so the cleanup threw `NullPointerException` out of the constructor, replaced the real cause, and never reached `super.abort()`, leaving the file behind anyway. That covers every append, blob and vector data file, and it happens on the existing `IOException` path as well.
   
   Both constructors now track whether opening completed and clean up from `finally`. `SingleFileWriter` does so through a private, non-overridable method, so no subclass state is touched while the super constructor is still running. The guard is `out != null || writer != null`: when `newOutputStream(path, false)` refuses to overwrite an existing file, nothing was created and that file is not ours to delete.
   
   Out of scope: a throw inside `SupportsDirectWrite.create` can still leave a file behind (Vortex creates it eagerly). That interface states the format has full control over its file IO, so the base class does not delete it.
   
   ### Tests
   
   New `SingleFileWriterTest` (8 cases) and `FormatTableSingleFileWriterTest` (3 cases). 6 of the 11 fail on master:
   
   - runtime exception while opening, for both `asyncWrite` values: the original exception surfaces and no file survives
   - a subclass whose `abort()` touches state assigned after `super(...)`: the real failure is not replaced by `NullPointerException`, checked on both the runtime-exception and the `IOException` path
   - failure after only the format writer was created: the writer is closed and the file deleted
   - regression guards: the `IOException` path still wraps into `UncheckedIOException`, an already existing file survives a failed open with its content intact, and a successful open leaves the stream open until `close()`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [core] Fix orphan file and NPE when a single file writer fails to open [paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi merged PR #8923:
URL: https://github.com/apache/paimon/pull/8923


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [core] Fix orphan file and NPE when a single file writer fails to open [paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi commented on PR #8923:
URL: https://github.com/apache/paimon/pull/8923#issuecomment-5130165034

   +1


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org