You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "dkp116 (via GitHub)" <gi...@apache.org> on 2026/07/13 17:08:02 UTC

[PR] GH 50351 - KeyValueMetadata::DeleteMany method crashes [arrow]

dkp116 opened a new pull request, #50500:
URL: https://github.com/apache/arrow/pull/50500

   ### Rationale for this change
   
   Investigated the two usages of `DeleteMany` in the codebase to verify whether there could be an issue caused by incorrect or duplicate indexes being passed.
   
   There are two locations where `DeleteMany` is called:
   
   1. `cpp/src/arrow/c/bridge.cc`
   
   ```cpp
   DeleteMany(metadata_.extension_name_index, metadata_.extension_serialized_index)
   ```
   
   The indexes are populated while iterating through metadata keys:
   
   ```cpp
   if (keys[i] == kExtensionTypeKeyName) {
     decoded.extension_name = values[i];
     decoded.extension_name_index = i;
   } else if (keys[i] == kExtensionMetadataKeyName) {
     decoded.extension_serialized = values[i];
     decoded.extension_serialized_index = i;
   }
   ```
   Here due to the if and else if statement there is not a case where they are the same value.
   
   
   
   2. `cpp/src/arrow/ipc/metadata_internal.cc`
   
   ```cpp
   metadata->DeleteMany({name_index, data_index});
   ```
   
   Here, the indexes are obtained using:
   
   ```cpp
   name_index = metadata->FindKey(kExtensionTypeKeyName);
   data_index = metadata->FindKey(kExtensionMetadataKeyName);
   ```
   
   `FindKey` traverses the `keys_` array maintained by `KeyValueMetadata` and returns the index of the matching key. Since the two searched keys are different:
   
   `cpp/src/arrow/extension_type.cc`
   
   ```cpp
   extern const char kExtensionTypeKeyName[] = "ARROW:extension:name";
   extern const char kExtensionMetadataKeyName[] = "ARROW:extension:metadata";
   ```
   
   the returned indexes refer to two different metadata entries.
   
   ### What changes are included in this PR?
   
    This change  documents/investigates the behavior of the existing `DeleteMany`,  and a log is produced when DeleteMany is accessing an invalid memory address instead of throwing with this error: 
   /usr/include/c++/16.1.1/bits/stl_vector.h:1253: constexpr std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](size_type) [with _Tp = std::__cxx11::basic_string<char>; _Alloc = std::allocator<std::__cxx11::basic_string<char> >; 
   reference = std::__cxx11::basic_string<char>&; size_type = long unsigned int]: Assertion '__n < this->size()' failed.
   
   Note: I have not prevented the use of the same index in DeleteMany as for example DeleteMany(1,1) will still give an output, given a valid key and array vector so no functionality has been changed, just a more readable log has been added.
   
   ### Are these changes tested?
   
   A test has been added to `cpp/src/arrow/util/key_value_metadata_test.cc` which checks to see the correct error is shown when an invalid memory address is being accessed
   
   ### Are there any user-facing changes?
   
   No user-facing changes.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50351 : [C++] KeyValueMetadata::DeleteMany method crashes [arrow]

Posted by "dkp116 (via GitHub)" <gi...@apache.org>.
dkp116 commented on code in PR #50500:
URL: https://github.com/apache/arrow/pull/50500#discussion_r4064736542


##########
cpp/src/arrow/util/key_value_metadata.cc:
##########
@@ -121,11 +124,23 @@ Status KeyValueMetadata::DeleteMany(std::vector<int64_t> indices) {
     ++shift;
     const auto start = indices[i] + 1;
     const auto stop = indices[i + 1];
-    DCHECK_GE(start, 0);
-    DCHECK_LE(start, size);
-    DCHECK_GE(stop, 0);
-    DCHECK_LE(stop, size);
+
+    if (ARROW_PREDICT_TRUE(start < 0 || start > size)) {
+      return Status::IndexError("KeyValueMetadata::DeleteMany: Start index ", start - 1,
+                                " out of bounds for metadata of size ", size);
+    }

Review Comment:
   corrected error handle 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50351 : [C++] KeyValueMetadata::DeleteMany method crashes [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #50500:
URL: https://github.com/apache/arrow/pull/50500#issuecomment-6217677917

   Could you check the following cases?
   
   1. Duplicated index after the 1st iteration such as `{0, 2, 2}`
   2. Multiple indices with `-1` such as `{-1, 0}`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50351 : [C++] KeyValueMetadata::DeleteMany method crashes [arrow]

Posted by "Reranko05 (via GitHub)" <gi...@apache.org>.
Reranko05 commented on code in PR #50500:
URL: https://github.com/apache/arrow/pull/50500#discussion_r4057633849


##########
cpp/src/arrow/util/key_value_metadata.cc:
##########
@@ -121,11 +124,23 @@ Status KeyValueMetadata::DeleteMany(std::vector<int64_t> indices) {
     ++shift;
     const auto start = indices[i] + 1;
     const auto stop = indices[i + 1];
-    DCHECK_GE(start, 0);
-    DCHECK_LE(start, size);
-    DCHECK_GE(stop, 0);
-    DCHECK_LE(stop, size);
+
+    if (ARROW_PREDICT_TRUE(start < 0 || start > size)) {
+      return Status::IndexError("KeyValueMetadata::DeleteMany: Start index ", start - 1,
+                                " out of bounds for metadata of size ", size);
+    }

Review Comment:
   nit: Can you use `ARROW_PREDICT_FALSE` for error branches. (Have not analysed the full code, just caught this.)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50351 : [C++] KeyValueMetadata::DeleteMany method crashes [arrow]

Posted by "Reranko05 (via GitHub)" <gi...@apache.org>.
Reranko05 commented on code in PR #50500:
URL: https://github.com/apache/arrow/pull/50500#discussion_r3637241089


##########
cpp/src/arrow/util/key_value_metadata.cc:
##########
@@ -126,6 +126,7 @@ Status KeyValueMetadata::DeleteMany(std::vector<int64_t> indices) {
     DCHECK_GE(stop, 0);
     DCHECK_LE(stop, size);
     for (int64_t index = start; index < stop; ++index) {
+      DCHECK_GE(index - shift, 0);

Review Comment:
   I noticed `DeleteMany` returns `Status`, but the function currently relies on DCHECKs for validating assumptions. Would it make sense to reject duplicate or otherwise invalid indices with `Status::Invalid` instead? That would provide runtime validation rather than only catching the issue in assertion-enabled builds.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50351: [C++] KeyValueMetadata::DeleteMany method crashes [arrow]

Posted by "dkp116 (via GitHub)" <gi...@apache.org>.
dkp116 commented on PR #50500:
URL: https://github.com/apache/arrow/pull/50500#issuecomment-6234499208

   > Could you check the following cases?
   > 
   >     1. Duplicated index after the 1st iteration such as `{0, 2, 2}`
   > 
   >     2. Multiple indices with `-1` such as `{-1, 0}`
   
   good catch, the -1,0 test case spotted an defect, added a fix for this and refactored out the validations as it was getting quite messy :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50351 : [C++] KeyValueMetadata::DeleteMany method crashes [arrow]

Posted by "Reranko05 (via GitHub)" <gi...@apache.org>.
Reranko05 commented on PR #50500:
URL: https://github.com/apache/arrow/pull/50500#issuecomment-6211553378

   @kou Could you review this when you have time?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50351: [C++] KeyValueMetadata::DeleteMany method crashes [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #50500:
URL: https://github.com/apache/arrow/pull/50500#issuecomment-6262941352

   I'm sorry, but why are we doing any of this? Before checking the "how", please first consider the "why".
   
   There is no reason to do extensive input validation for every utility method in Arrow. This will be wasteful and difficult to maintain.
   
   IOW, we can just specify that the input to `DeleteMany` should not contain any duplicate values.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH 50351 - KeyValueMetadata::DeleteMany method crashes [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #50500:
URL: https://github.com/apache/arrow/pull/50500#issuecomment-4960570162

   <!--
     Licensed to the Apache Software Foundation (ASF) under one
     or more contributor license agreements.  See the NOTICE file
     distributed with this work for additional information
     regarding copyright ownership.  The ASF licenses this file
     to you under the Apache License, Version 2.0 (the
     "License"); you may not use this file except in compliance
     with the License.  You may obtain a copy of the License at
   
       http://www.apache.org/licenses/LICENSE-2.0
   
     Unless required by applicable law or agreed to in writing,
     software distributed under the License is distributed on an
     "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
     KIND, either express or implied.  See the License for the
     specific language governing permissions and limitations
     under the License.
   -->
   
   Thanks for opening a pull request!
   
   If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
   
   Opening GitHub issues ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
   
   Then could you also rename the pull request title in the following format?
   
       GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
   
   or
   
       MINOR: [${COMPONENT}] ${SUMMARY}
   
   See also:
   
     * [Other pull requests](https://github.com/apache/arrow/pulls/)
     * [Contribution Guidelines - Contributing Overview](https://arrow.apache.org/docs/developers/overview.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50351 : [C++] KeyValueMetadata::DeleteMany method crashes [arrow]

Posted by "dkp116 (via GitHub)" <gi...@apache.org>.
dkp116 commented on code in PR #50500:
URL: https://github.com/apache/arrow/pull/50500#discussion_r4057284547


##########
cpp/src/arrow/util/key_value_metadata.cc:
##########
@@ -126,6 +126,7 @@ Status KeyValueMetadata::DeleteMany(std::vector<int64_t> indices) {
     DCHECK_GE(stop, 0);
     DCHECK_LE(stop, size);
     for (int64_t index = start; index < stop; ++index) {
+      DCHECK_GE(index - shift, 0);

Review Comment:
   I have amended so Status::IndexError is returned instead of DCHECKS being used 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org