You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "and2049 (via GitHub)" <gi...@apache.org> on 2026/09/15 06:04:00 UTC
[PR] fix: relax validation for missing validity buffer when null_count is -1 [arrow-nanoarrow]
and2049 opened a new pull request, #942:
URL: https://github.com/apache/arrow-nanoarrow/pull/942
Allow a missing validity buffer when the null count is unknown, matching
Arrow C++’s import behavior. Arrays reporting a positive null count still
require a validity buffer.
Add regression coverage for unknown null counts with a missing validity
buffer, including sliced arrays and positive-null-count rejection.
Fixes #855
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: relax validation for missing validity buffer when null_count is -1 [arrow-nanoarrow]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #942:
URL: https://github.com/apache/arrow-nanoarrow/pull/942#issuecomment-5683647669
## [Codecov](https://app.codecov.io/gh/apache/arrow-nanoarrow/pull/942?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 78.19%. Comparing base ([`b896d19`](https://app.codecov.io/gh/apache/arrow-nanoarrow/commit/b896d19e4afc0b519bec808003257cf24bf3ebe9?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`c0eb1d9`](https://app.codecov.io/gh/apache/arrow-nanoarrow/commit/c0eb1d99a998717e08cb8e488ac5509120aaa560?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #942 +/- ##
=======================================
Coverage 78.19% 78.19%
=======================================
Files 106 106
Lines 16843 16843
Branches 1985 1985
=======================================
Hits 13170 13170
Misses 2439 2439
Partials 1234 1234
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/arrow-nanoarrow/pull/942?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: relax validation for missing validity buffer when null_count is -1 [arrow-nanoarrow]
Posted by "and2049 (via GitHub)" <gi...@apache.org>.
and2049 commented on code in PR #942:
URL: https://github.com/apache/arrow-nanoarrow/pull/942#discussion_r4018585628
##########
src/nanoarrow/common/array_test.cc:
##########
@@ -2441,6 +2441,70 @@ TEST(ArrayTest, ArrayViewTestBasic) {
ArrowArrayViewReset(&array_view);
}
+TEST(ArrayTest, ArrayViewTestMissingValidityUnknownNullCount) {
+ struct ArrowError error;
+
+ // A missing validity buffer with null_count == -1 is valid
+ int32_t values[] = {11, 12, 13};
+ const void* buffers[] = {nullptr, values};
+
+ nanoarrow::UniqueArray array;
+ array->length = 2;
+ array->offset = 1;
+ array->null_count = -1;
+ array->n_buffers = 2;
+ array->n_children = 0;
+ array->buffers = buffers;
+ array->children = nullptr;
+ array->dictionary = nullptr;
+ array->release = [](struct ArrowArray*) {};
+
+ nanoarrow::UniqueArrayView array_view;
+ ArrowArrayViewInitFromType(array_view.get(), NANOARROW_TYPE_INT32);
+ ASSERT_EQ(ArrowArrayViewSetArray(array_view.get(), array.get(), &error), NANOARROW_OK)
+ << error.message;
+ EXPECT_EQ(
+ ArrowArrayViewValidate(array_view.get(), NANOARROW_VALIDATION_LEVEL_FULL, &error),
+ NANOARROW_OK)
+ << error.message;
+ EXPECT_EQ(ArrowArrayViewGetBufferView(array_view.get(), 0).size_bytes, 0);
+ EXPECT_EQ(ArrowArrayViewComputeNullCount(array_view.get()), 0);
+ EXPECT_EQ(ArrowArrayViewIsNull(array_view.get(), 0), 0);
+ EXPECT_EQ(ArrowArrayViewIsNull(array_view.get(), 1), 0);
+ EXPECT_EQ(ArrowArrayViewGetIntUnsafe(array_view.get(), 0), 12);
+ EXPECT_EQ(ArrowArrayViewGetIntUnsafe(array_view.get(), 1), 13);
+
+ // A missing validity buffer with null_count > 0 is still an error
+ array->null_count = 1;
+ EXPECT_EQ(ArrowArrayViewSetArray(array_view.get(), array.get(), &error), EINVAL);
+ EXPECT_STREQ(error.message,
+ "Expected int32 array buffer 0 to have size >= 1 bytes but found buffer "
+ "with 0 bytes");
+
+ // Same for a string array (["a", "bb", "ccc"])
+ int32_t offsets[] = {0, 1, 3, 6};
+ const void* string_buffers[] = {nullptr, offsets, "abbccc"};
+ array->null_count = -1;
+ array->n_buffers = 3;
+ array->buffers = string_buffers;
+
+ nanoarrow::UniqueArrayView string_view;
+ ArrowArrayViewInitFromType(string_view.get(), NANOARROW_TYPE_STRING);
+ ASSERT_EQ(ArrowArrayViewSetArray(string_view.get(), array.get(), &error), NANOARROW_OK)
+ << error.message;
+ EXPECT_EQ(
+ ArrowArrayViewValidate(string_view.get(), NANOARROW_VALIDATION_LEVEL_FULL, &error),
+ NANOARROW_OK)
+ << error.message;
+ EXPECT_EQ(ArrowArrayViewGetBufferView(string_view.get(), 0).size_bytes, 0);
+ EXPECT_EQ(ArrowArrayViewComputeNullCount(string_view.get()), 0);
+ EXPECT_EQ(ArrowArrayViewGetStringUnsafe(string_view.get(), 0), "bb"_asv);
+ EXPECT_EQ(ArrowArrayViewGetStringUnsafe(string_view.get(), 1), "ccc"_asv);
+
+ array->null_count = 1;
+ EXPECT_EQ(ArrowArrayViewSetArray(string_view.get(), array.get(), &error), EINVAL);
Review Comment:
I've removed the string-array case, and the test now uses the same int32 array and only changes the null count.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: relax validation for missing validity buffer when null_count is -1 [arrow-nanoarrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on code in PR #942:
URL: https://github.com/apache/arrow-nanoarrow/pull/942#discussion_r4018309985
##########
src/nanoarrow/common/array_test.cc:
##########
@@ -2441,6 +2441,70 @@ TEST(ArrayTest, ArrayViewTestBasic) {
ArrowArrayViewReset(&array_view);
}
+TEST(ArrayTest, ArrayViewTestMissingValidityUnknownNullCount) {
+ struct ArrowError error;
+
+ // A missing validity buffer with null_count == -1 is valid
+ int32_t values[] = {11, 12, 13};
+ const void* buffers[] = {nullptr, values};
+
+ nanoarrow::UniqueArray array;
+ array->length = 2;
+ array->offset = 1;
+ array->null_count = -1;
+ array->n_buffers = 2;
+ array->n_children = 0;
+ array->buffers = buffers;
+ array->children = nullptr;
+ array->dictionary = nullptr;
+ array->release = [](struct ArrowArray*) {};
+
+ nanoarrow::UniqueArrayView array_view;
+ ArrowArrayViewInitFromType(array_view.get(), NANOARROW_TYPE_INT32);
+ ASSERT_EQ(ArrowArrayViewSetArray(array_view.get(), array.get(), &error), NANOARROW_OK)
+ << error.message;
+ EXPECT_EQ(
+ ArrowArrayViewValidate(array_view.get(), NANOARROW_VALIDATION_LEVEL_FULL, &error),
+ NANOARROW_OK)
+ << error.message;
+ EXPECT_EQ(ArrowArrayViewGetBufferView(array_view.get(), 0).size_bytes, 0);
+ EXPECT_EQ(ArrowArrayViewComputeNullCount(array_view.get()), 0);
+ EXPECT_EQ(ArrowArrayViewIsNull(array_view.get(), 0), 0);
+ EXPECT_EQ(ArrowArrayViewIsNull(array_view.get(), 1), 0);
+ EXPECT_EQ(ArrowArrayViewGetIntUnsafe(array_view.get(), 0), 12);
+ EXPECT_EQ(ArrowArrayViewGetIntUnsafe(array_view.get(), 1), 13);
+
+ // A missing validity buffer with null_count > 0 is still an error
+ array->null_count = 1;
+ EXPECT_EQ(ArrowArrayViewSetArray(array_view.get(), array.get(), &error), EINVAL);
+ EXPECT_STREQ(error.message,
+ "Expected int32 array buffer 0 to have size >= 1 bytes but found buffer "
+ "with 0 bytes");
+
+ // Same for a string array (["a", "bb", "ccc"])
+ int32_t offsets[] = {0, 1, 3, 6};
+ const void* string_buffers[] = {nullptr, offsets, "abbccc"};
+ array->null_count = -1;
+ array->n_buffers = 3;
+ array->buffers = string_buffers;
+
+ nanoarrow::UniqueArrayView string_view;
+ ArrowArrayViewInitFromType(string_view.get(), NANOARROW_TYPE_STRING);
+ ASSERT_EQ(ArrowArrayViewSetArray(string_view.get(), array.get(), &error), NANOARROW_OK)
+ << error.message;
+ EXPECT_EQ(
+ ArrowArrayViewValidate(string_view.get(), NANOARROW_VALIDATION_LEVEL_FULL, &error),
+ NANOARROW_OK)
+ << error.message;
+ EXPECT_EQ(ArrowArrayViewGetBufferView(string_view.get(), 0).size_bytes, 0);
+ EXPECT_EQ(ArrowArrayViewComputeNullCount(string_view.get()), 0);
+ EXPECT_EQ(ArrowArrayViewGetStringUnsafe(string_view.get(), 0), "bb"_asv);
+ EXPECT_EQ(ArrowArrayViewGetStringUnsafe(string_view.get(), 1), "ccc"_asv);
+
+ array->null_count = 1;
+ EXPECT_EQ(ArrowArrayViewSetArray(string_view.get(), array.get(), &error), EINVAL);
Review Comment:
Can you to use the exact same array as above and just modify the null count? I don't think we need to use a different type here to cover the changed lines.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: relax validation for missing validity buffer when null_count is -1 [arrow-nanoarrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot merged PR #942:
URL: https://github.com/apache/arrow-nanoarrow/pull/942
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org