You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "lriggs (via GitHub)" <gi...@apache.org> on 2026/06/09 23:18:40 UTC

[PR] GH-50140 Fix castVARCHAR(decimal128) native memory corruption / SIGSEGV on allocation failure [arrow]

lriggs opened a new pull request, #50141:
URL: https://github.com/apache/arrow/pull/50141

   # [C++][Gandiva] Fix castVARCHAR(decimal128) native memory corruption / SIGSEGV on allocation failure
   
   ### Rationale for this change
   
   The Gandiva `castVARCHAR_decimal128_int64` path could corrupt native memory and
   crash the process (SIGSEGV) when the output-string arena allocation failed
   (e.g. `CAST(decimal AS VARCHAR)` under memory pressure). Three independent
   defects combined to cause this:
   
   1. The `castVARCHAR` decimal128 registry entry was missing
      `NativeFunction::kCanReturnErrors`, so generated code skipped the error check
      and ignored any error the function reported.
   2. `gdv_fn_dec_to_string` set the output length to a positive value *before*
      checking whether the allocation succeeded, then returned `nullptr` — leaving
      the caller to copy from an invalid buffer with a positive length.
   3. `castVARCHAR_decimal128_int64` did not validate a negative requested output
      length and did not handle an upstream allocation failure.
   
   ### What changes are included in this PR?
   
   - **`function_registry_string.cc`**: Add `NativeFunction::kCanReturnErrors` to the
     `castVARCHAR` `decimal128` entry so the generated code checks for and
     propagates errors instead of assuming the function never fails.
   
   - **`gdv_function_stubs.cc`** (`gdv_fn_dec_to_string`): Only write the output
     length *after* a successful allocation. On allocation failure, set
     `*dec_str_len = 0` and return an empty string so callers never copy from an
     invalid buffer using a stale, positive length.
   
   - **`precompiled/decimal_wrapper.cc`** (`castVARCHAR_decimal128_int64`):
     - Reject a negative output length with a graceful error
       (`"Output buffer length can't be negative"`) instead of using it as a copy
       size.
     - Bail out safely (zero length, empty string) if the upstream
       `gdv_fn_dec_to_string` call failed, since the error has already been set.
   
   - **`tests/decimal_test.cc`**: Add `TestCastVarCharDecimalNegativeLength`, a
     regression test that casts a decimal to varchar with a negative output length
     and asserts the query fails gracefully with the expected error message rather
     than crashing. This also exercises the `kCanReturnErrors` flag — without it the
     error would not propagate and the test would fail.
   
   ### Behavior change
   
   Queries such as `CAST(decimal AS VARCHAR)` that previously crashed the process
   (SIGSEGV) under memory pressure now fail gracefully with an error message about
   the allocation failure / invalid length, and the rest of the system is
   unaffected.
   
   ### Are these changes tested?
   
   Yes, unit tests.
   
   ### Are there any user-facing changes?
   
   No.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50140: [C++][Gandiva] Fix castVARCHAR(decimal128) native memory corruption / SIGSEGV on allocation failure [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou merged PR #50141:
URL: https://github.com/apache/arrow/pull/50141


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50140: [C++][Gandiva] Fix castVARCHAR(decimal128) native memory corruption / SIGSEGV on allocation failure [arrow]

Posted by "lriggs (via GitHub)" <gi...@apache.org>.
lriggs commented on code in PR #50141:
URL: https://github.com/apache/arrow/pull/50141#discussion_r3391955764


##########
cpp/src/gandiva/precompiled/decimal_wrapper.cc:
##########
@@ -423,11 +423,23 @@ FORCE_INLINE
 char* castVARCHAR_decimal128_int64(int64_t context, int64_t x_high, uint64_t x_low,
                                    int32_t x_precision, int32_t x_scale,
                                    int64_t out_len_param, int32_t* out_length) {
+  if (out_len_param < 0) {
+    gdv_fn_context_set_error_msg(context, "Output buffer length can't be negative");
+    *out_length = 0;
+    return const_cast<char*>("");
+  }
   int32_t full_dec_str_len;
   char* dec_str =
       gdv_fn_dec_to_string(context, x_high, x_low, x_scale, &full_dec_str_len);
-  int32_t trunc_dec_str_len =
-      out_len_param < full_dec_str_len ? out_len_param : full_dec_str_len;
+  if (dec_str == nullptr) {
+    // Allocation failed upstream; error message is already set. Avoid copying from
+    // an invalid buffer with a non-zero length.
+    *out_length = 0;

Review Comment:
   Yes, it will be null. This nullptr is set when the memory allocation for the output string fails. The null part is actually handled at another level using the validity bits.
   
   null decimal input is handled correctly, and it's actually cleaner now than before.
   
   Tracing the slow path (now active because we added kCanReturnErrors) in llvm_generator.cc:797-832:
   
   For a null input, is_valid is false, so BuildIfElse takes the else_lambda (lines 821-829):
   
   castVARCHAR_decimal128_int64 is not called at all.
   It returns a dummy value: else_value = NullConstant(...) and, because utf8() is binary-like, else_value_len = i32_constant(0).
   Separately, the result validity is computed by the validity dex as the AND of the input validities (kResultNullIfNull, per expr_decomposer.cc:92. A null input → result validity bit = 0 → the output row is marked null, and the dummy value/length are masked out.
   
   So the consumer sees a proper null. Two things worth noting:
   
   Correctness was the same before our fix — the fast path also produced a null output for null rows (via the validity mask); the only difference was that it ran the function on garbage x_high/x_low and threw the result away. Now it's genuinely skipped.
   
   No risk from the 0-length dummy — the else branch returns a null pointer with length 0, and since the row is invalid in the bitmap, nothing reads it. This is unrelated to the arena 0-size behavior we just discussed (no arena_malloc call happens on this path at all).
   
   So: null input → function skipped → output null, exactly as expected.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50140: [C++][Gandiva] Fix castVARCHAR(decimal128) native memory corruption / SIGSEGV on allocation failure [arrow]

Posted by "selvaganesang (via GitHub)" <gi...@apache.org>.
selvaganesang commented on code in PR #50141:
URL: https://github.com/apache/arrow/pull/50141#discussion_r3390276191


##########
cpp/src/gandiva/precompiled/decimal_wrapper.cc:
##########
@@ -423,11 +423,23 @@ FORCE_INLINE
 char* castVARCHAR_decimal128_int64(int64_t context, int64_t x_high, uint64_t x_low,

Review Comment:
   The function name is kind of misleading. Intuitively, I thought that we are casting varchar into decimal. It looks like it is converting decimal and int64 to varchar.



##########
cpp/src/gandiva/precompiled/decimal_wrapper.cc:
##########
@@ -423,11 +423,23 @@ FORCE_INLINE
 char* castVARCHAR_decimal128_int64(int64_t context, int64_t x_high, uint64_t x_low,
                                    int32_t x_precision, int32_t x_scale,
                                    int64_t out_len_param, int32_t* out_length) {
+  if (out_len_param < 0) {
+    gdv_fn_context_set_error_msg(context, "Output buffer length can't be negative");
+    *out_length = 0;
+    return const_cast<char*>("");
+  }
   int32_t full_dec_str_len;
   char* dec_str =
       gdv_fn_dec_to_string(context, x_high, x_low, x_scale, &full_dec_str_len);
-  int32_t trunc_dec_str_len =
-      out_len_param < full_dec_str_len ? out_len_param : full_dec_str_len;
+  if (dec_str == nullptr) {
+    // Allocation failed upstream; error message is already set. Avoid copying from
+    // an invalid buffer with a non-zero length.
+    *out_length = 0;

Review Comment:
   Will the result be NULL? Curious Is this case of  CAST((CAST(NULL AS DECIMAL) AS VARCHAR) ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50140: [C++][Gandiva] Fix castVARCHAR(decimal128) native memory corruption / SIGSEGV on allocation failure [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #50141:
URL: https://github.com/apache/arrow/pull/50141#issuecomment-5185161101

   Oh, sorry. I missed this. I'll merge this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50140: [C++][Gandiva] Fix castVARCHAR(decimal128) native memory corruption / SIGSEGV on allocation failure [arrow]

Posted by "lriggs (via GitHub)" <gi...@apache.org>.
lriggs commented on PR #50141:
URL: https://github.com/apache/arrow/pull/50141#issuecomment-5185140308

   @kou Any chance this can be merged?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50140 Fix castVARCHAR(decimal128) native memory corruption / SIGSEGV on allocation failure [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #50141:
URL: https://github.com/apache/arrow/pull/50141#issuecomment-4664968854

   :warning: GitHub issue #50140 **has been automatically assigned in GitHub** to PR creator.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-50140: [C++][Gandiva] Fix castVARCHAR(decimal128) native memory corruption / SIGSEGV on allocation failure [arrow]

Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #50141:
URL: https://github.com/apache/arrow/pull/50141#issuecomment-5187606464

   After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 22a2b51f763a297829abf57a68f8e788ceda4b59.
   
   There were no benchmark performance regressions. 🎉
   
   The [full Conbench report](https://github.com/apache/arrow/runs/92209159638) has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org