You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "mapleFU (via GitHub)" <gi...@apache.org> on 2023/10/07 10:31:53 UTC

[I] [C++][Parquet] Debug Build might fail in `DictEncoder` dtor checking [arrow]

mapleFU opened a new issue, #38117:
URL: https://github.com/apache/arrow/issues/38117

   ### Describe the enhancement requested
   
   This is the case described in https://github.com/apache/arrow/pull/35520 . But I didn't find it when previous meet the bug because I just use release build.
   
   ```c++
     void Close() override {
       if (!closed_) {
         closed_ = true;
         CheckRowsWritten();
   
         // Avoid invalid state if ColumnWriter::Close() throws internally.
         auto column_writers = std::move(column_writers_);
         for (size_t i = 0; i < column_writers.size(); i++) {
           if (column_writers[i]) {
             total_bytes_written_ += column_writers[i]->Close();
             total_compressed_bytes_written_ +=
                 column_writers[i]->total_compressed_bytes_written();
           }
         }
   
         // Ensures all columns have been written
         metadata_->set_num_rows(num_rows_);
         metadata_->Finish(total_bytes_written_, row_group_ordinal_);
       }
     }
   ```
   
   When close file with 10 columns, assume column writer 2 has exception on `Close`, it throw an exception. and the third one is a `DictEncoder`, so it would `DCHECK` failed:
   
   ```c++
     ~DictEncoderImpl() override { DCHECK(buffered_indices_.empty()); }
   ```
   
   ### Component(s)
   
   C++, Parquet


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Debug Build might fail in `DictEncoder` dtor checking [arrow]

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #38117:
URL: https://github.com/apache/arrow/issues/38117#issuecomment-1751740630

   @pitrou I've draft a basic fixing for this https://github.com/apache/arrow/pull/38118
   
   I change `DCHECK` to `ARROW_LOG(WARNING)`. This might not the best, because it might cause user forget to flush the `DictEncoder`. Another way is provide an `abort`.
   
   ```
         // Avoid invalid state if ColumnWriter::Close() throws internally.
         auto column_writers = std::move(column_writers_);
         for (size_t i = 0; i < column_writers.size(); i++) {
           if (column_writers[i]) {
              column_writers[i]->abort(); // abort might not throw exception.
           }
         }
         for (size_t i = 0; i < column_writers.size(); i++) {
           if (column_writers[i]) {
             total_bytes_written_ += column_writers[i]->Close();
             ...
           }
         }
   ```
   
   And change `DCHECK` to:
   
   ```c++
   DCHECK(aborted_ || !buffered_indices_.empty());
   ```
   
   Any idea is ok for me :-)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Debug Build might fail in `DictEncoder` dtor checking [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou closed issue #38117: [C++][Parquet] Debug Build might fail in `DictEncoder` dtor checking
URL: https://github.com/apache/arrow/issues/38117


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org