You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/11/10 16:11:44 UTC

[GitHub] [arrow] pitrou opened a new pull request #8626: ARROW-10545: [C++] Fix crash on invalid Parquet file (OSS-Fuzz)

pitrou opened a new pull request #8626:
URL: https://github.com/apache/arrow/pull/8626


   Also removed a memory allocation (probably not performance-critical).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on pull request #8626: ARROW-10545: [C++] Fix crash on invalid Parquet file (OSS-Fuzz)

Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #8626:
URL: https://github.com/apache/arrow/pull/8626#issuecomment-724804154


   Note the ASAN CI tests will probably fail until #8617 is merged.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on pull request #8626: ARROW-10545: [C++] Fix crash on invalid Parquet file (OSS-Fuzz)

Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #8626:
URL: https://github.com/apache/arrow/pull/8626#issuecomment-725298582


   As I said above, this will break CI until #8617 is merged.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] github-actions[bot] commented on pull request #8626: ARROW-10545: [C++] Fix crash on invalid Parquet file (OSS-Fuzz)

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8626:
URL: https://github.com/apache/arrow/pull/8626#issuecomment-724813055


   https://issues.apache.org/jira/browse/ARROW-10545


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] emkornfield commented on a change in pull request #8626: ARROW-10545: [C++] Fix crash on invalid Parquet file (OSS-Fuzz)

Posted by GitBox <gi...@apache.org>.
emkornfield commented on a change in pull request #8626:
URL: https://github.com/apache/arrow/pull/8626#discussion_r520732790



##########
File path: cpp/src/parquet/level_conversion.cc
##########
@@ -35,16 +36,17 @@ namespace internal {
 namespace {
 
 using ::arrow::internal::CpuInfo;
+using ::arrow::util::optional;
 
 template <typename OffsetType>
 void DefRepLevelsToListInfo(const int16_t* def_levels, const int16_t* rep_levels,
                             int64_t num_def_levels, LevelInfo level_info,
                             ValidityBitmapInputOutput* output, OffsetType* offsets) {
   OffsetType* orig_pos = offsets;
-  std::unique_ptr<::arrow::internal::FirstTimeBitmapWriter> valid_bits_writer;
+  optional<::arrow::internal::FirstTimeBitmapWriter> valid_bits_writer;
   if (output->valid_bits) {
-    valid_bits_writer.reset(new ::arrow::internal::FirstTimeBitmapWriter(
-        output->valid_bits, output->valid_bits_offset, num_def_levels));
+    valid_bits_writer.emplace(output->valid_bits, output->valid_bits_offset,
+                              output->values_read_upper_bound);

Review comment:
       :( nice catch.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a change in pull request #8626: ARROW-10545: [C++] Fix crash on invalid Parquet file (OSS-Fuzz)

Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #8626:
URL: https://github.com/apache/arrow/pull/8626#discussion_r520685686



##########
File path: cpp/src/parquet/level_conversion.cc
##########
@@ -35,16 +36,17 @@ namespace internal {
 namespace {
 
 using ::arrow::internal::CpuInfo;
+using ::arrow::util::optional;
 
 template <typename OffsetType>
 void DefRepLevelsToListInfo(const int16_t* def_levels, const int16_t* rep_levels,
                             int64_t num_def_levels, LevelInfo level_info,
                             ValidityBitmapInputOutput* output, OffsetType* offsets) {
   OffsetType* orig_pos = offsets;
-  std::unique_ptr<::arrow::internal::FirstTimeBitmapWriter> valid_bits_writer;
+  optional<::arrow::internal::FirstTimeBitmapWriter> valid_bits_writer;
   if (output->valid_bits) {
-    valid_bits_writer.reset(new ::arrow::internal::FirstTimeBitmapWriter(
-        output->valid_bits, output->valid_bits_offset, num_def_levels));
+    valid_bits_writer.emplace(output->valid_bits, output->valid_bits_offset,
+                              output->values_read_upper_bound);

Review comment:
       The bitmap writer length was wrong here. @emkornfield 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] emkornfield commented on pull request #8626: ARROW-10545: [C++] Fix crash on invalid Parquet file (OSS-Fuzz)

Posted by GitBox <gi...@apache.org>.
emkornfield commented on pull request #8626:
URL: https://github.com/apache/arrow/pull/8626#issuecomment-724844425


   LGTM, feel free to merge, or I'll do it when I'm on a computer that i can do it from.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kou closed pull request #8626: ARROW-10545: [C++] Fix crash on invalid Parquet file (OSS-Fuzz)

Posted by GitBox <gi...@apache.org>.
kou closed pull request #8626:
URL: https://github.com/apache/arrow/pull/8626


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org