You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "mapleFU (via GitHub)" <gi...@apache.org> on 2023/05/09 16:47:58 UTC

[GitHub] [arrow] mapleFU opened a new pull request, #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

mapleFU opened a new pull request, #35520:
URL: https://github.com/apache/arrow/pull/35520

   <!--
   Thanks for opening a pull request!
   If this is your first pull request you can find detailed information on how 
   to contribute here:
     * [New Contributor's Guide](https://arrow.apache.org/docs/dev/developers/guide/step_by_step/pr_lifecycle.html#reviews-and-merge-of-the-pull-request)
     * [Contributing Overview](https://arrow.apache.org/docs/dev/developers/overview.html)
   
   
   If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
   
   Opening GitHub issues ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
   
   Then could you also rename the pull request title in the following format?
   
       GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
   
   or
   
       MINOR: [${COMPONENT}] ${SUMMARY}
   
   In the case of PARQUET issues on JIRA the title also supports:
   
       PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
   
   -->
   
   ### Rationale for this change
   
   <!--
    Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.  
   -->
   
   Mentioned in https://github.com/apache/arrow/issues/35519
   
   ### What changes are included in this PR?
   
   Exception handling in `CheckRowsWritten`
   
   <!--
   There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
   -->
   
   ### Are these changes tested?
   
   <!--
   We typically require tests for all PRs in order to:
   1. Prevent the code from being accidentally broken by subsequent changes
   2. Serve as another way to document the expected behavior of the code
   
   If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
   -->
   
   No, I don't know how to mock it currently
   
   ### Are there any user-facing changes?
   
   no
   
   <!--
   If there are user-facing changes then we may require documentation to be updated before approving the PR.
   -->
   
   <!--
   If there are any breaking changes to public APIs, please uncomment the line below and explain which changes are breaking.
   -->
   <!-- **This PR includes breaking changes to public APIs.** -->
   
   <!--
   Please uncomment the line below (and provide explanation) if the changes fix either (a) a security vulnerability, (b) a bug that caused incorrect or invalid data to be produced, or (c) a bug that causes a crash (even when the API contract is upheld). We use this to highlight fixes to issues that may affect users without their knowledge. For this reason, fixing bugs that cause errors don't count, since those are usually obvious.
   -->
   <!-- **This PR contains a "Critical Fix".** -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] mapleFU commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1544528972

   @pitrou Seems R failed is not related. Would you mind take a look?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] mapleFU commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1542201770

   Comment resolved. I tried to solve some CI problems here: https://github.com/apache/arrow/pull/35527
   
   Seems CI failed frequently these days...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] pitrou commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1542429645

   We can't merge until we make sure the R CI failure is unrelated.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] github-actions[bot] commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1540524739

   :warning: GitHub issue #35519 **has been automatically assigned in GitHub** to PR creator.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] mapleFU commented on a diff in pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on code in PR #35520:
URL: https://github.com/apache/arrow/pull/35520#discussion_r1189485658


##########
cpp/src/parquet/file_writer.cc:
##########
@@ -219,17 +219,17 @@ class RowGroupSerializer : public RowGroupWriter::Contents {
       closed_ = true;
       CheckRowsWritten();
 
-      for (size_t i = 0; i < column_writers_.size(); i++) {
-        if (column_writers_[i]) {
-          total_bytes_written_ += column_writers_[i]->Close();
+      // If ColumnWriter::Close thrown an exception, avoid
+      // have bad states in column_writers_.

Review Comment:
   it's already closed, whether success or not, no any write can comes to `RowGroupSerializer`. I guess it will cause `FileSerializer` destruct



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] mapleFU commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1540524686

   @pitrou Would you mind take a look? Analyze is in https://github.com/apache/arrow/issues/35519#issue-1702385078


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] pitrou commented on a diff in pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #35520:
URL: https://github.com/apache/arrow/pull/35520#discussion_r1188921194


##########
cpp/src/parquet/file_writer.cc:
##########
@@ -380,15 +382,15 @@ class FileSerializer : public ParquetFileWriter::Contents {
   void AddKeyValueMetadata(
       const std::shared_ptr<const KeyValueMetadata>& key_value_metadata) override {
     if (key_value_metadata_ == nullptr) {
-      key_value_metadata_ = std::move(key_value_metadata);
+      key_value_metadata_ = key_value_metadata;
     } else if (key_value_metadata != nullptr) {
       key_value_metadata_ = key_value_metadata_->Merge(*key_value_metadata);
     }
   }
 
   ~FileSerializer() override {
     try {
-      Close();
+      FileSerializer::Close();

Review Comment:
   Why is this change required?



##########
cpp/src/parquet/file_writer.cc:
##########
@@ -380,15 +382,15 @@ class FileSerializer : public ParquetFileWriter::Contents {
   void AddKeyValueMetadata(
       const std::shared_ptr<const KeyValueMetadata>& key_value_metadata) override {
     if (key_value_metadata_ == nullptr) {
-      key_value_metadata_ = std::move(key_value_metadata);
+      key_value_metadata_ = key_value_metadata;

Review Comment:
   Good catch!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] pitrou merged pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou merged PR #35520:
URL: https://github.com/apache/arrow/pull/35520


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] pitrou commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1540546202

   I think we should instead prevent `RowGroupSerializer::Close` from creating an invalid internal state. For example:
   ```c++
     void Close() override {
       if (!closed_) {
         closed_ = true;
         CheckRowsWritten();
   
         auto column_writers = std::move(columns_writers_);
         for (const auto writer : column_writers) {
           if (writer) {
             total_bytes_written_ += writer->Close();
             total_compressed_bytes_written_ +=
                 writer->total_compressed_bytes_written();
           }
         }
   
         // Ensures all columns have been written
         metadata_->set_num_rows(num_rows_);
         metadata_->Finish(total_bytes_written_, row_group_ordinal_);
       }
     }
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] pitrou commented on a diff in pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #35520:
URL: https://github.com/apache/arrow/pull/35520#discussion_r1188920715


##########
cpp/src/parquet/file_writer.cc:
##########
@@ -219,17 +219,17 @@ class RowGroupSerializer : public RowGroupWriter::Contents {
       closed_ = true;
       CheckRowsWritten();
 
-      for (size_t i = 0; i < column_writers_.size(); i++) {
-        if (column_writers_[i]) {
-          total_bytes_written_ += column_writers_[i]->Close();
+      // If ColumnWriter::Close thrown an exception, avoid
+      // have bad states in column_writers_.
+      auto column_writers = std::move(column_writers_);
+      for (size_t i = 0; i < column_writers.size(); i++) {
+        if (column_writers[i]) {
+          total_bytes_written_ += column_writers[i]->Close();
           total_compressed_bytes_written_ +=
               column_writers_[i]->total_compressed_bytes_written();

Review Comment:
   ```suggestion
                 column_writers[i]->total_compressed_bytes_written();
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] pitrou commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1548071183

   I'm restarting CI a last time but it looks like this is ready to go.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] pitrou commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1542502456

   Hmm, actually the segfault is already known:
   https://github.com/apache/arrow/issues/35391
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] pitrou commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1542380300

   > Seems CI failed frequently these days...
   
   Unfortunately :-( Are there issues already open for these failures?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] mapleFU commented on a diff in pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on code in PR #35520:
URL: https://github.com/apache/arrow/pull/35520#discussion_r1188922582


##########
cpp/src/parquet/file_writer.cc:
##########
@@ -380,15 +382,15 @@ class FileSerializer : public ParquetFileWriter::Contents {
   void AddKeyValueMetadata(
       const std::shared_ptr<const KeyValueMetadata>& key_value_metadata) override {
     if (key_value_metadata_ == nullptr) {
-      key_value_metadata_ = std::move(key_value_metadata);
+      key_value_metadata_ = key_value_metadata;
     } else if (key_value_metadata != nullptr) {
       key_value_metadata_ = key_value_metadata_->Merge(*key_value_metadata);
     }
   }
 
   ~FileSerializer() override {
     try {
-      Close();
+      FileSerializer::Close();

Review Comment:
   Just tiny style change. `Close` is a virtual function, but calling virtual function in dtor is not "virtual", so change it calling `FileSerializer::Close()` explicit.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] pitrou commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1542410594

   Hmm, the R CI failure may be related as it seems to succeed on git main:
   https://github.com/apache/arrow/actions/runs/4937417890/jobs/8826087049
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] mapleFU commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1542504423

   😭Hurt by so many failed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] ursabot commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "ursabot (via GitHub)" <gi...@apache.org>.
ursabot commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1549137274

   Benchmark runs are scheduled for baseline = e2e3a9df28db03b09b5a83a60fa8293dfef6d0d8 and contender = 2d76d9a526f9827283bb7dfac60715b6ad4aec34. 2d76d9a526f9827283bb7dfac60715b6ad4aec34 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/6ed12a58585a470f869849b391ebfcc8...9f29eac47c5f42e280a11b3b8e802b90/)
   [Finished :arrow_down:0.26% :arrow_up:0.0%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/047ee94172d440ad86e5b435d1d5f493...506d4c05219f4eb1a0afea814e6e5d1c/)
   [Finished :arrow_down:0.51% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/25dc661ddcc7437f836227a1724ba395...033aa70395684296afe5b85e29f59b88/)
   [Finished :arrow_down:0.42% :arrow_up:0.03%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/893e7cf799da4cf7b6d570f782cd6566...7e3d0019d80c4c11a66086e7b2d30a52/)
   Buildkite builds:
   [Finished] [`2d76d9a5` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/2875)
   [Finished] [`2d76d9a5` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/2911)
   [Finished] [`2d76d9a5` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/2876)
   [Finished] [`2d76d9a5` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/2901)
   [Finished] [`e2e3a9df` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/2874)
   [Finished] [`e2e3a9df` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/2910)
   [Finished] [`e2e3a9df` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/2875)
   [Finished] [`e2e3a9df` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/2900)
   Supported benchmarks:
   ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
   test-mac-arm: Supported benchmark langs: C++, Python, R
   ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
   ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] mapleFU commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1542426194

   Yeah, I don't want to rebase this because there are still too much failed tests :-(
   Just wait others idea on this patch or merge if you think it's ok


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] github-actions[bot] commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1540524684

   * Closes: #35519


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] wgtmac commented on a diff in pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on code in PR #35520:
URL: https://github.com/apache/arrow/pull/35520#discussion_r1189477737


##########
cpp/src/parquet/file_writer.cc:
##########
@@ -219,17 +219,17 @@ class RowGroupSerializer : public RowGroupWriter::Contents {
       closed_ = true;
       CheckRowsWritten();
 
-      for (size_t i = 0; i < column_writers_.size(); i++) {
-        if (column_writers_[i]) {
-          total_bytes_written_ += column_writers_[i]->Close();
+      // If ColumnWriter::Close thrown an exception, avoid
+      // have bad states in column_writers_.

Review Comment:
   ```suggestion
         // Avoid invalid state if ColumnWriter::Close() throws internally.
   ```



##########
cpp/src/parquet/file_writer.cc:
##########
@@ -219,17 +219,17 @@ class RowGroupSerializer : public RowGroupWriter::Contents {
       closed_ = true;
       CheckRowsWritten();
 
-      for (size_t i = 0; i < column_writers_.size(); i++) {
-        if (column_writers_[i]) {
-          total_bytes_written_ += column_writers_[i]->Close();
+      // If ColumnWriter::Close thrown an exception, avoid
+      // have bad states in column_writers_.

Review Comment:
   If anything wrong happens here, should we let the FileSerializer know that it cannot accept any write any more?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] mapleFU commented on a diff in pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on code in PR #35520:
URL: https://github.com/apache/arrow/pull/35520#discussion_r1189664088


##########
cpp/src/parquet/file_writer.cc:
##########
@@ -219,17 +219,17 @@ class RowGroupSerializer : public RowGroupWriter::Contents {
       closed_ = true;
       CheckRowsWritten();
 
-      for (size_t i = 0; i < column_writers_.size(); i++) {
-        if (column_writers_[i]) {
-          total_bytes_written_ += column_writers_[i]->Close();
+      // If ColumnWriter::Close thrown an exception, avoid
+      // have bad states in column_writers_.

Review Comment:
   ```c++
     void Close() override {
       if (!closed_) {
         closed_ = true;
         CheckRowsWritten();
    
   
         // ...
   
         // Ensures all columns have been written
         metadata_->set_num_rows(num_rows_);
         metadata_->Finish(total_bytes_written_, row_group_ordinal_);
       }
     }
   ```
   
   I guess it will mark `closed_` first and throw exception. During stack unwind, the `RowGroupSerializer` will destruct. And during next time `Close` is called, it will do nothing



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] mapleFU commented on a diff in pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on code in PR #35520:
URL: https://github.com/apache/arrow/pull/35520#discussion_r1188922582


##########
cpp/src/parquet/file_writer.cc:
##########
@@ -380,15 +382,15 @@ class FileSerializer : public ParquetFileWriter::Contents {
   void AddKeyValueMetadata(
       const std::shared_ptr<const KeyValueMetadata>& key_value_metadata) override {
     if (key_value_metadata_ == nullptr) {
-      key_value_metadata_ = std::move(key_value_metadata);
+      key_value_metadata_ = key_value_metadata;
     } else if (key_value_metadata != nullptr) {
       key_value_metadata_ = key_value_metadata_->Merge(*key_value_metadata);
     }
   }
 
   ~FileSerializer() override {
     try {
-      Close();
+      FileSerializer::Close();

Review Comment:
   Just tiny style change. `Close` is a virtual function, but calling virtual function in dtor is not "virtual", so change it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] mapleFU commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1540573471

   @pitrou Good Idea! Fix as your advice


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] wgtmac commented on a diff in pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on code in PR #35520:
URL: https://github.com/apache/arrow/pull/35520#discussion_r1189506915


##########
cpp/src/parquet/file_writer.cc:
##########
@@ -219,17 +219,17 @@ class RowGroupSerializer : public RowGroupWriter::Contents {
       closed_ = true;
       CheckRowsWritten();
 
-      for (size_t i = 0; i < column_writers_.size(); i++) {
-        if (column_writers_[i]) {
-          total_bytes_written_ += column_writers_[i]->Close();
+      // If ColumnWriter::Close thrown an exception, avoid
+      // have bad states in column_writers_.

Review Comment:
   This is true if the exception happens when calling `FileSerializer::Close()`. What if this happens when the user directly calls `RowGroupWriter::Close()` to close the current row group and starts a new one?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] mapleFU commented on pull request #35520: GH-35519: [C++][Parquet] Fixing exception handling in parquet FileSerializer

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35520:
URL: https://github.com/apache/arrow/pull/35520#issuecomment-1544451699

   ```
   [ RUN      ] TestScan/TestCsvFileFormatScan.ScanBatchSize/0Threaded16b1024r
   unknown file: error: SEH exception with code 0xc00000fd thrown in the test body.
   [  FAILED  ] TestScan/TestCsvFileFormatScan.ScanBatchSize/0Threaded16b1024r, where GetParam() = Threaded16b1024r (314 ms)
   ```
   
   I guess this is unrelated. https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/47021836


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org