You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "mapleFU (via GitHub)" <gi...@apache.org> on 2023/06/19 09:04:17 UTC

[GitHub] [arrow] mapleFU opened a new pull request, #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

mapleFU opened a new pull request, #35694:
URL: https://github.com/apache/arrow/pull/35694

   <!--
   Thanks for opening a pull request!
   If this is your first pull request you can find detailed information on how 
   to contribute here:
     * [New Contributor's Guide](https://arrow.apache.org/docs/dev/developers/guide/step_by_step/pr_lifecycle.html#reviews-and-merge-of-the-pull-request)
     * [Contributing Overview](https://arrow.apache.org/docs/dev/developers/overview.html)
   
   
   If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
   
   Opening GitHub issues ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
   
   Then could you also rename the pull request title in the following format?
   
       GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
   
   or
   
       MINOR: [${COMPONENT}] ${SUMMARY}
   
   In the case of PARQUET issues on JIRA the title also supports:
   
       PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
   
   -->
   
   ### Rationale for this change
   
   <!--
    Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.  
   -->
   
   
   1. Parquet will regard `FixedSizeList` as regular list, and write rep-levels and def-levels if possible
   2. When read from Parquet, if no null (or nested null, like `FixedSizeList` doesn't has null but it parent has) is contained, the file can be read correctly
   3. However, when read from Parquet with nullable, the `AssembleArray` will not work well. It takes continuous "child array" and child offsets, and need to reserve extra padding space for "null". Previous reader doesn't implement it.
   
   To be more clear:
   
   ```
   for FixedSizeList(2): [[0, 0], null]
   
   data to be Assemble Array:
   null: [0 1]
   data: [0 0]
   offsets: [0, 2, 2]
   
   required output:
   null: [0 1]
   data: [0, 0, undefined, undefined]
   ```
   
   previous reader doesn't implement that required output
   
   ### What changes are included in this PR?
   
   <!--
   There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
   -->
   
   Rebuild the `FixedSizeList` when having null member
   
   ### Are these changes tested?
   
   <!--
   We typically require tests for all PRs in order to:
   1. Prevent the code from being accidentally broken by subsequent changes
   2. Serve as another way to document the expected behavior of the code
   
   If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
   -->
   
   Yes
   
   ### Are there any user-facing changes?
   
   No
   
   <!--
   If there are user-facing changes then we may require documentation to be updated before approving the PR.
   -->
   
   <!--
   If there are any breaking changes to public APIs, please uncomment the line below and explain which changes are breaking.
   -->
   <!-- **This PR includes breaking changes to public APIs.** -->
   
   <!--
   Please uncomment the line below (and provide explanation) if the changes fix either (a) a security vulnerability, (b) a bug that caused incorrect or invalid data to be produced, or (c) a bug that causes a crash (even when the API contract is upheld). We use this to highlight fixes to issues that may affect users without their knowledge. For this reason, fixing bugs that cause errors don't count, since those are usually obvious.
   -->
   <!-- **This PR contains a "Critical Fix".** -->
   * Closes: #35692


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1560338177

   I'll close this and submit another patch using `null_slot_usage`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1596728191

   I found `null_slot_usage` can be efficient, but does it needs to hack into `ReadRecordData`? Since that `ReadRecordData` will handles the leaf underlying buffer.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1560488182

   Emmm I'm sure it works, `null_slot_usage` will be more efficient because it doesn't copy more data, but my solution may "copy" the underlying leaf data. But I think my solution is right. Should you keep this as a quick fixing? I'm a bit busy this week, and may only have time for `null_slot_usage` fixing this weekend. @pitrou @emkornfield 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] emkornfield commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1557664803

   @mapleFU its not clear to me that this approach works for all level of nestings.  When I was thinking about the problem, I thought it would be better to populate and make use of [null_slot_usage](https://github.com/apache/arrow/blob/main/cpp/src/parquet/level_conversion.h#L53) on reading each array back.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1634532914

   https://github.com/apache/arrow/pull/35694#issuecomment-1596728191
   @emkornfield @pitrou would you mind help for this? When we have:
   1. `FixedSizeList<FixedSizeList<int, 2>, 2>`
   2.`FixedSizeList<List<FixedSizeList<int, 2>, 2>>`
   
   What should  `null_slot_usage` be? Should it be? Should them be:
   1. `(2)` for `FixedSizeList<int, 2>`, and `(4)` for `FixedSizeList<FixedSizeList<int, 2>, 2>`?
   2. `(2)` for `FixedSizeList<int, 2>`, and `(4)` for `FixedSizeList<FixedSizeList<int, 2>, 2>`, what would be for `FixedSizeList<List<FixedSizeList<int, 2>, 2>>`?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] spenczar commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: tiny fixing FixedSizeListReader::AssembleArray

Posted by "spenczar (via GitHub)" <gi...@apache.org>.
spenczar commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1555145609

   Reading https://arrow.apache.org/docs/format/Columnar.html#fixed-size-list-layout more closely, I'm not so sure that the zero-sized elements are correct:
    
   > A fixed size list type is specified like FixedSizeList<T>[N], where T is any type (primitive or nested) and N is a 32-bit signed integer representing the length of the lists.
   >
   > A fixed size list array is represented by a values array, which is a child array of type T. T may also be a nested type. The value in slot j of a fixed size list array is stored in an N-long slice of the values array, starting at an offset of j * N.
   
   `j * N` makes it sound like offsets are always fixed size for fixed size lists. In fact, that document makes it sound like there shouldn't be an offsets buffer at all, since fixed-size lists use "slots". 
   
   Could this actually be a writer bug?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] emkornfield commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1560461941

   If this code works (and we don't think null_slot_usage can be more efficient) it is ok to pursue this approach, so feel free to reopen


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] emkornfield commented on a diff in pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on code in PR #35694:
URL: https://github.com/apache/arrow/pull/35694#discussion_r1200917743


##########
cpp/src/parquet/arrow/arrow_reader_writer_test.cc:
##########
@@ -3030,25 +3051,48 @@ TEST(ArrowReadWrite, FixedSizeList) {
   CheckSimpleRoundtrip(table, 2, props_store_schema);
 }
 
-TEST(ArrowReadWrite, ListOfStructOfList2) {
+TEST(ArrowReadWrite, FixedSizeList2) {
   using ::arrow::field;
-  using ::arrow::list;
+  using ::arrow::fixed_size_list;
   using ::arrow::struct_;
 
-  auto type =
-      list(field("item",
-                 struct_({field("a", ::arrow::int16(), /*nullable=*/false),
-                          field("b", list(::arrow::int64()), /*nullable=*/false)}),
-                 /*nullable=*/false));
+  auto type = fixed_size_list(::arrow::int16(), /*size=*/3);
 
   const char* json = R"([
-      [{"a": 123, "b": [1, 2, 3]}],
       null,
-      [],
-      [{"a": 456, "b": []}, {"a": 789, "b": [null]}, {"a": 876, "b": [4, 5, 6]}]])";
+      [null, 1, 2],
+      [null, 3, 4],
+      null,
+      [5, null, 6],
+      [7, null, 8],
+      null,
+      [9, 10, null],
+      null,
+      null,
+      [11, 12, 13],
+      [14, 15, 16]])";
   auto array = ::arrow::ArrayFromJSON(type, json);
   auto table = ::arrow::Table::Make(::arrow::schema({field("root", type)}), {array});
-  CheckSimpleRoundtrip(table, 2);
+  auto props_store_schema = ArrowWriterProperties::Builder().store_schema()->build();
+  CheckSimpleRoundtrip(table, 2, props_store_schema);

Review Comment:
   I think we need more tests here that can cover nulls in the middle and multiple levels of nested fixed sizes lists with other types of lists.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1558428174

   I got what you mean, I didn't know `null_slot_usage` previously, thanks a lot!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-3547075809

   Thank you for your contribution. Unfortunately, this pull request has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this PR will be closed in 14 days.  Feel free to re-open this if it has been closed in error. If you do not have repository permissions to reopen the PR, please tag a maintainer.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1556233354

   I've found another unrelated issue when writing unittests: https://github.com/apache/arrow/issues/35697


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] github-actions[bot] commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: tiny fixing FixedSizeListReader::AssembleArray

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1555062170

   :warning: GitHub issue #35692 **has been automatically assigned in GitHub** to PR creator.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] github-actions[bot] commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: tiny fixing FixedSizeListReader::AssembleArray

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1555062062

   * Closes: #35692


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1555621089

   @spenczar Would you mind have a try on this patch? The code might not be the final commit code, but I want to make sure that problem got fixed :-)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU commented on a diff in pull request #35694: GH-35692: [C++][Parquet] Parquet: tiny fixing FixedSizeListReader::AssembleArray

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on code in PR #35694:
URL: https://github.com/apache/arrow/pull/35694#discussion_r1199329895


##########
cpp/src/parquet/arrow/reader.cc:
##########
@@ -673,13 +675,39 @@ class PARQUET_NO_EXPORT FixedSizeListReader : public ListReader<int32_t> {
     DCHECK_EQ(field()->type()->id(), ::arrow::Type::FIXED_SIZE_LIST);
     const auto& type = checked_cast<::arrow::FixedSizeListType&>(*field()->type());
     const int32_t* offsets = reinterpret_cast<const int32_t*>(data->buffers[1]->data());
+    // Need to reassemble data with correct offsets and paddings.
+    if (field()->nullable() && data->null_count != 0) {
+      std::unique_ptr<::arrow::ArrayBuilder> temp_builder;
+      ARROW_RETURN_NOT_OK(::arrow::MakeBuilder(::arrow::default_memory_pool(),
+                                               field()->type(), &temp_builder));
+      auto fixed_sized_list_builder =
+          checked_cast<::arrow::FixedSizeListBuilder*>(temp_builder.get());
+      for (int x = 1; x <= data->length; x++) {

Review Comment:
   (I guess using validity_bitmap is better here, I may change it after I wake up)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: tiny fixing FixedSizeListReader::AssembleArray

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1555161271

   @spenczar Hi, I think it's just a reader bug. The written data is not corrupt.
   
   1. Parquet will regard `FixedSizeList` as regular list, and write rep-levels and def-levels if possible
   2. When read from Parquet, if no null (or nested null, like `FixedSizeList` doesn't has null but it parent has) is contained, the file can be read correctly
   3. However, when read from Parquet with nullable, the `AssembleArray` will not work well. It takes continuous "child array" and child offsets, and need to reserve extra padding space for "null". Previous reader doesn't implement it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU closed pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU closed pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList
URL: https://github.com/apache/arrow/pull/35694


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1558453319

   ```
     if (valid_bits_writer.has_value()) {
       valid_bits_writer->Finish();
     }
     if (offsets != nullptr) {
       output->values_read = offsets - orig_pos;
     } else if (valid_bits_writer.has_value()) {
       output->values_read = valid_bits_writer->position();
     }
     if (output->null_count > 0 && level_info.null_slot_usage > 1) {
       throw ParquetException(
           "Null values with null_slot_usage > 1 not supported."
           "(i.e. FixedSizeLists with null values are not supported)");
     }
   ```
   
   Seems that I should support the handling logic here?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1556869260

   @emkornfield Can you take a look here? I'm not sure I understand the issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: tiny fixing FixedSizeListReader::AssembleArray

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1555162787

   I've submit a fixing, but I'm not familiar with `FixedSizeListArrayBuilder` and `Array`, so I may made some mistakes. Can @pitrou helps me review this patch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] spenczar commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "spenczar (via GitHub)" <gi...@apache.org>.
spenczar commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1670491622

   Is there any way I can help get some motion here? This remains a pretty nasty blocker for some applications of Arrow and Parquet for us. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1671894170

   I believe this patch will solve part of the problem. However, it might has some performance lost, since it need to copy the underlying data. Feel free to use this as a patch
   
   Using `null_slot` might be better here. But I need some help to tell me how to implement it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mapleFU commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1557668511

   I've tested the nested case. But I can take a look at `null_slot_usage`, thanks a lot!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] emkornfield commented on pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList

Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on PR #35694:
URL: https://github.com/apache/arrow/pull/35694#issuecomment-1557735255

   To be clear this was not plumbed through to actually work but that was the intent of the field.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #35694: GH-35692: [C++][Parquet] Parquet: fixing FixedSizeListReader::AssembleArray when has null FixedSizeList
URL: https://github.com/apache/arrow/pull/35694


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org