You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "xiedeyantu (via GitHub)" <gi...@apache.org> on 2026/05/27 13:36:34 UTC

[PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

xiedeyantu opened a new pull request, #22564:
URL: https://github.com/apache/datafusion/pull/22564

   ## Which issue does this PR close?
   - Closes #22261 .
   
   ## Rationale for this change
   The current `log` implementation does not handle `log(0)` consistently across all execution paths. In some cases it can return infinity-like results instead of a clear error. This change makes zero input fail explicitly so the behavior is predictable and easier to reason about.
   
   ## What changes are included in this PR?
   This change adds zero-value validation in log.rs and applies it across the floating-point and decimal log code paths. Any input value equal to zero now returns `cannot take logarithm of zero` instead of continuing with a calculation that would produce an unexpected result.
   
   The regression test in scalar.slt was also updated to expect the new error behavior for `log(0)`.
   
   ## Are these changes tested?
   Yes. The sqllogictest coverage was updated to verify the new `log(0)` error behavior. 
   
   ## Are there any user-facing changes?
   Yes. `log(0)` now returns an explicit error with the message `cannot take logarithm of zero` instead of producing an infinite result.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#issuecomment-4603436293

   @Jefffrey I added `log(1, 64)` back and made sure its error message matched that of duckdb. I removed the tests in simplification.rs that wouldn't be simplified because they conflicted with each other; I don't think these tests need to be kept anyway.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#discussion_r3342018982


##########
datafusion/functions/src/math/log.rs:
##########
@@ -358,43 +386,83 @@ impl ScalarUDFImpl for LogFunc {
         } else {
             lit(ScalarValue::new_ten(&number_datatype)?)
         };
+        let base_datatype = info.get_data_type(&base)?;
+
+        if is_zero_literal(&number, &number_datatype)?
+            || is_zero_literal(&base, &base_datatype)?
+        {
+            return Ok(ExprSimplifyResult::Original(original_log_args(
+                num_args, &base, &number,
+            )?));
+        }
+
+        let base_is_valid_literal = is_valid_log_base_literal(&base)?;
 
         match number {
             Expr::Literal(value, _)
-                if value == ScalarValue::new_one(&number_datatype)? =>
+                if value == ScalarValue::new_one(&number_datatype)?
+                    && base_is_valid_literal =>
             {
                 Ok(ExprSimplifyResult::Simplified(lit(ScalarValue::new_zero(
                     &info.get_data_type(&base)?,
                 )?)))
             }
             Expr::ScalarFunction(ScalarFunction { func, mut args })
-                if is_pow(&func) && args.len() == 2 && base == args[0] =>
+                if is_pow(&func)
+                    && args.len() == 2
+                    && base == args[0]
+                    && base_is_valid_literal =>

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on code in PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#discussion_r3332284248


##########
datafusion/functions/src/math/log.rs:
##########
@@ -358,43 +386,83 @@ impl ScalarUDFImpl for LogFunc {
         } else {
             lit(ScalarValue::new_ten(&number_datatype)?)
         };
+        let base_datatype = info.get_data_type(&base)?;
+
+        if is_zero_literal(&number, &number_datatype)?
+            || is_zero_literal(&base, &base_datatype)?
+        {
+            return Ok(ExprSimplifyResult::Original(original_log_args(
+                num_args, &base, &number,
+            )?));
+        }
+
+        let base_is_valid_literal = is_valid_log_base_literal(&base)?;
 
         match number {
             Expr::Literal(value, _)
-                if value == ScalarValue::new_one(&number_datatype)? =>
+                if value == ScalarValue::new_one(&number_datatype)?
+                    && base_is_valid_literal =>
             {
                 Ok(ExprSimplifyResult::Simplified(lit(ScalarValue::new_zero(
                     &info.get_data_type(&base)?,
                 )?)))
             }
             Expr::ScalarFunction(ScalarFunction { func, mut args })
-                if is_pow(&func) && args.len() == 2 && base == args[0] =>
+                if is_pow(&func)
+                    && args.len() == 2
+                    && base == args[0]
+                    && base_is_valid_literal =>

Review Comment:
   I apologize; this was indeed my oversight. I originally intended to temporarily refrain from simplifying the reverse operations of log and power, but I missed it in my tightening logic. I'd like to ask if it would be better for us to temporarily abandon the simplification of these reverse operations? Because I feel that we seem unable to know the true range of values ​​for a column (or an expression).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew commented on code in PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#discussion_r3338184839


##########
datafusion/functions/src/math/log.rs:
##########
@@ -358,43 +386,83 @@ impl ScalarUDFImpl for LogFunc {
         } else {
             lit(ScalarValue::new_ten(&number_datatype)?)
         };
+        let base_datatype = info.get_data_type(&base)?;
+
+        if is_zero_literal(&number, &number_datatype)?
+            || is_zero_literal(&base, &base_datatype)?
+        {
+            return Ok(ExprSimplifyResult::Original(original_log_args(
+                num_args, &base, &number,
+            )?));
+        }
+
+        let base_is_valid_literal = is_valid_log_base_literal(&base)?;
 
         match number {
             Expr::Literal(value, _)
-                if value == ScalarValue::new_one(&number_datatype)? =>
+                if value == ScalarValue::new_one(&number_datatype)?
+                    && base_is_valid_literal =>
             {
                 Ok(ExprSimplifyResult::Simplified(lit(ScalarValue::new_zero(
                     &info.get_data_type(&base)?,
                 )?)))
             }
             Expr::ScalarFunction(ScalarFunction { func, mut args })
-                if is_pow(&func) && args.len() == 2 && base == args[0] =>
+                if is_pow(&func)
+                    && args.len() == 2
+                    && base == args[0]
+                    && base_is_valid_literal =>

Review Comment:
   I think we should temporarily abandon these reverse-op simplifications. We do not know the true runtime range of columns or arbitrary expressions, and the rewrite can remove validation that should happen at execution time. Safer to only simplify literal/fully constant cases, or skip the rewrite unless non-zero can be proven.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew commented on code in PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#discussion_r3328063647


##########
datafusion/functions/src/math/log.rs:
##########


Review Comment:
   Thanks for adding the zero-literal guard. I think there's still a gap here.
   
   The new check is added below at lines 405-417, but this `log(a, power(a, b)) => b` rewrite runs first. That means an expression like `log(0, power(0, 2))` can be simplified directly to `2` without ever calling `invoke_with_args`, even though the value argument is zero and should produce `cannot take logarithm of zero`.
   
   Could we make sure the zero-value validation runs before rewrites that can eliminate the log expression, or alternatively have this rewrite reject cases where the base or value can be proven to be zero? Otherwise the zero-value invariant can still be bypassed through simplification.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#issuecomment-4620820720

   > > > the scope of this PR seems to have extended quite a bit beyond the original intention
   > > 
   > > 
   > > Yes, I asked above whether to fix the error log(1, 64), and it is now included, so I'm not sure how to define the scope of this PR.
   > 
   > my suggestion was to defer such fixes to a later PR
   
   Ok, then I'll revert to the original version, which includes your other suggested changes, but excludes the version tested with log(1, 64). Is that alright?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on code in PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#discussion_r3354693153


##########
datafusion/functions/src/math/log.rs:
##########
@@ -71,6 +66,11 @@ impl Default for LogFunc {
 
 impl LogFunc {
     pub fn new() -> Self {
+        let as_decimal = Coercion::new_implicit(

Review Comment:
   sorry why is this change being made now? we're now casting integers to decimal?



##########
datafusion/functions/src/math/log.rs:
##########
@@ -106,58 +101,158 @@ fn is_valid_integer_base(base: f64) -> bool {
     base.trunc() == base && base >= 2.0 && base <= u32::MAX as f64
 }
 
+#[inline]
+fn exact_integer_base(base: f64) -> Option<u32> {
+    is_valid_integer_base(base).then_some(base as u32)
+}
+
+#[inline]
+fn exact_integer_log(value: u128, base: u32) -> Option<u32> {
+    if value == 0 {
+        return None;
+    }
+
+    let base_u128 = u128::from(base);
+    let int_log = value.ilog(base_u128);
+    (base_u128.checked_pow(int_log) == Some(value)).then_some(int_log)
+}
+
+#[inline]
+fn exact_positive_u128_from_float<T: Float + ToPrimitive>(value: T) -> Option<u128> {
+    let value_u128 = value.to_u128()?;
+    if value_u128 == 0 {
+        return None;
+    }
+
+    let exact_value = T::from(value_u128)?;
+    (value == exact_value).then_some(value_u128)
+}

Review Comment:
   same goes here, why are these changes being made here now?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew commented on code in PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#discussion_r3324384347


##########
datafusion/functions/src/math/log.rs:
##########
@@ -247,27 +269,33 @@ impl ScalarUDFImpl for LogFunc {
         let value = value.to_array(args.number_rows)?;
 
         let output: ArrayRef = match value.data_type() {
-            DataType::Float16 => {
-                calculate_binary_math::<Float16Type, Float16Type, Float16Type, _>(
-                    &value,
-                    &base,
-                    |value, base| Ok(value.log(base)),
-                )?
-            }
-            DataType::Float32 => {
-                calculate_binary_math::<Float32Type, Float32Type, Float32Type, _>(
-                    &value,
-                    &base,
-                    |value, base| Ok(value.log(base)),
-                )?
-            }
-            DataType::Float64 => {
-                calculate_binary_math::<Float64Type, Float64Type, Float64Type, _>(
-                    &value,
-                    &base,
-                    |value, base| Ok(value.log(base)),
-                )?
-            }
+            DataType::Float16 => calculate_binary_math::<

Review Comment:
   Small cleanup suggestion: the three float branches now repeat the same validation and log pattern.
   
   It might be worth using a small helper for the callback body, something like `validate_log_value(value as f64)?; Ok(value.log(base))`, if that can be made generic over the float types cleanly. If not, the current duplication seems fine to me.
   
   
   



##########
datafusion/functions/src/math/log.rs:
##########


Review Comment:
   In addition,  I think the new zero-value invariant is not being enforced end-to-end yet because expression simplification can bypass `invoke_with_args`.
   
   For example, `SELECT log(0, 0)` still returns `1.0` through the existing `log(a, a) => 1` rewrite, even though the value argument is zero and should now fail with `cannot take logarithm of zero`.
   
   I verified this after rebuilding `datafusion-cli`: `select log(0,0);` returns `1.0`, while `select log(0);` and `select log(2,0);` both error as expected.
   
   Could you please either make the simplification rules reject zero literals for the value argument, or enforce this invariant before or inside simplification and constant folding? It would also be good to add regression coverage for the binary optimized path.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#issuecomment-4608234868

   @Jefffrey @kosiew I've discovered several issues with the original implementation of the log function, such as the log(1, 64) example you mentioned. I'd like to ask if we should fix all of these in this single PR? Currently, all boundary conditions are not being considered, and it seems we need a major refactoring. Therefore, I'd like to ask whether we should create a new PR for a large refactoring, or fix the issues one by one?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#issuecomment-4620782617

   > > the scope of this PR seems to have extended quite a bit beyond the original intention
   > 
   > Yes, I asked above whether to fix the error log(1, 64), and it is now included, so I'm not sure how to define the scope of this PR.
   
   my suggestion was to defer such fixes to a later PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#issuecomment-4620577446

   > the scope of this PR seems to have extended quite a bit beyond the original intention
   
   Yes, I asked above whether to fix the error log(1, 64), and it is now included, so I'm not sure how to define the scope of this PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu closed pull request #22564: Fix log(0.0::float8) should error, not return -inf
URL: https://github.com/apache/datafusion/pull/22564


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on code in PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#discussion_r3341282096


##########
datafusion/functions/src/math/log.rs:
##########
@@ -157,7 +174,20 @@ fn log_decimal128(value: i128, scale: i8, base: f64) -> Result<f64, ArrowError>
             return Ok(int_log as f64);
         }
     }
-    decimal_to_f64(value, scale).map(|v| v.log(base))
+    decimal_to_f64(value, scale).and_then(|v| {
+        validate_log_value(v)?;
+        Ok(v.log(base))
+    })
+}
+
+/// Compute logarithm for Float16, Float32, and Float64 values
+#[inline]
+fn compute_float_log<T: Float + ToPrimitive>(value: T, base: T) -> Result<T, ArrowError> {
+    let value_f64 = value.to_f64().ok_or_else(|| {
+        ArrowError::ComputeError("Cannot convert value to f64".to_string())
+    })?;

Review Comment:
   we can avoid this conversion to f64 by using something like [`is_zero()`](https://docs.rs/num-traits/latest/num_traits/identities/trait.Zero.html#tymethod.is_zero)



##########
datafusion/functions/src/math/log.rs:
##########
@@ -358,43 +386,83 @@ impl ScalarUDFImpl for LogFunc {
         } else {
             lit(ScalarValue::new_ten(&number_datatype)?)
         };
+        let base_datatype = info.get_data_type(&base)?;
+
+        if is_zero_literal(&number, &number_datatype)?
+            || is_zero_literal(&base, &base_datatype)?
+        {
+            return Ok(ExprSimplifyResult::Original(original_log_args(
+                num_args, &base, &number,
+            )?));
+        }
+
+        let base_is_valid_literal = is_valid_log_base_literal(&base)?;
 
         match number {
             Expr::Literal(value, _)
-                if value == ScalarValue::new_one(&number_datatype)? =>
+                if value == ScalarValue::new_one(&number_datatype)?
+                    && base_is_valid_literal =>
             {
                 Ok(ExprSimplifyResult::Simplified(lit(ScalarValue::new_zero(
                     &info.get_data_type(&base)?,
                 )?)))
             }
             Expr::ScalarFunction(ScalarFunction { func, mut args })
-                if is_pow(&func) && args.len() == 2 && base == args[0] =>
+                if is_pow(&func)
+                    && args.len() == 2
+                    && base == args[0]
+                    && base_is_valid_literal =>
             {
                 let b = args.pop().unwrap(); // length checked above
                 Ok(ExprSimplifyResult::Simplified(b))
             }
             number => {
-                if number == base {
+                if number == base && base_is_valid_literal {
                     Ok(ExprSimplifyResult::Simplified(lit(ScalarValue::new_one(
                         &number_datatype,
                     )?)))
                 } else {
-                    let args = match num_args {
-                        1 => vec![number],
-                        2 => vec![base, number],
-                        _ => {
-                            return internal_err!(
-                                "Unexpected number of arguments in log::simplify"
-                            );
-                        }
-                    };
-                    Ok(ExprSimplifyResult::Original(args))
+                    Ok(ExprSimplifyResult::Original(original_log_args(
+                        num_args, &base, &number,
+                    )?))
                 }
             }
         }
     }
 }
 
+#[inline]
+fn original_log_args(num_args: usize, base: &Expr, number: &Expr) -> Result<Vec<Expr>> {
+    match num_args {
+        1 => Ok(vec![number.clone()]),
+        2 => Ok(vec![base.clone(), number.clone()]),
+        _ => {
+            internal_err!("Unexpected number of arguments in log::simplify")
+        }
+    }
+}
+
+#[inline]
+fn is_zero_literal(expr: &Expr, data_type: &DataType) -> Result<bool> {
+    match expr {
+        Expr::Literal(value, _) => Ok(*value == ScalarValue::new_zero(data_type)?),

Review Comment:
   ```suggestion
           Expr::Literal(value, _) => Ok(*value == ScalarValue::new_zero(&value.data_type())?),
   ```



##########
datafusion/functions/src/math/log.rs:
##########
@@ -358,43 +386,83 @@ impl ScalarUDFImpl for LogFunc {
         } else {
             lit(ScalarValue::new_ten(&number_datatype)?)
         };
+        let base_datatype = info.get_data_type(&base)?;
+
+        if is_zero_literal(&number, &number_datatype)?
+            || is_zero_literal(&base, &base_datatype)?
+        {
+            return Ok(ExprSimplifyResult::Original(original_log_args(

Review Comment:
   its probably easier to just keep a clone of the original `args` to reuse here instead of having a separate function to reconstruct it



##########
datafusion/sqllogictest/test_files/scalar.slt:
##########


Review Comment:
   we're getting rid of the `log(1, 64)` test case here? can we preserve this



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on code in PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#discussion_r3346144341


##########
datafusion/functions/src/math/log.rs:
##########
@@ -106,10 +101,37 @@ fn is_valid_integer_base(base: f64) -> bool {
     base.trunc() == base && base >= 2.0 && base <= u32::MAX as f64
 }
 
+#[inline]
+fn validate_log_value(value: f64) -> Result<(), ArrowError> {
+    if value == 0.0 {
+        Err(ArrowError::ComputeError(
+            "cannot take logarithm of zero".to_string(),
+        ))
+    } else {
+        Ok(())
+    }
+}
+
+#[inline]
+fn validate_log_base(base: f64) -> Result<(), ArrowError> {
+    if base < 0.0 {
+        Err(ArrowError::ComputeError(
+            "cannot take logarithm of a negative number".to_string(),
+        ))
+    } else if base == 1.0 {
+        Err(ArrowError::ComputeError(
+            "division by zero in based logarithm".to_string(),
+        ))

Review Comment:
   We should combine these checks with `compute_float_log` since we're duplicating them; I feel we can easily turn these functions generic over `Float` and then reuse them within `compute_float_log` below



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#issuecomment-4612533607

   @Jefffrey @kosiew I've refactored the log implementation based on my existing capabilities. Could you please take another look?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#issuecomment-4581625305

   @kosiew Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew commented on code in PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#discussion_r3330164811


##########
datafusion/functions/src/math/log.rs:
##########


Review Comment:
   I think the same issue applies to the `log(a, a) -> 1` simplification.
   
   For example:
   
   ```sql
   select log(column1, column1)
   from (values (0), (2)) as t(column1);
   ```
   
   After simplification this returns:
   
   ```text
   1.0
   1.0
   ```
   
   but the `a = 0` row would normally evaluate `log(0, 0)` and should raise the new zero-value error.
   
   Could we gate this rewrite on proving the value is non-zero, or otherwise preserve the runtime validation path? It would be great to add a regression test that exercises a column value of zero rather than only literal-zero cases.



##########
datafusion/functions/src/math/log.rs:
##########


Review Comment:
   Nice improvement on the literal zero handling. I think there is still a gap for non-literal expressions.
   
   The guard only catches zero literals before simplification. For expressions like `log(a, power(a, b))`, the rewrite can still eliminate the `log` call entirely. That means rows where `a = 0` never reach `invoke_with_args`, so the new `cannot take logarithm of zero` validation is skipped.
   
   For example:
   
   ```sql
   select log(column1, power(column1, 2))
   from (values (0), (2)) as t(column1);
   ```
   
   After the rewrite this returns:
   
   ```text
   2.0
   2.0
   ```
   
   but without the rewrite, the first row evaluates `power(0, 2) = 0` and should raise the zero-logarithm error.
   
   Could we either avoid this rewrite unless the value can be proven non-zero, or preserve the runtime validation for rows that evaluate to zero? A regression test with a column containing a zero row would help cover this case.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "xiedeyantu (via GitHub)" <gi...@apache.org>.
xiedeyantu commented on PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#issuecomment-4589443383

   > @xiedeyantu Thanks for the iteration. There is still one correctness issue where the rewrites can bypass the runtime zero-value validation for non-literal expressions.
   
   @kosiew Thank you very much for your verification and review. I've reconsidered it now, and I think our simplify logic needs some adjustments. Perhaps we could intelligently support simplification in two modes: constant or a combination of constant and expression. So I've made some adjustments; could you please take another look and see if it's acceptable?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew commented on code in PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#discussion_r3332260030


##########
datafusion/functions/src/math/log.rs:
##########
@@ -358,43 +386,83 @@ impl ScalarUDFImpl for LogFunc {
         } else {
             lit(ScalarValue::new_ten(&number_datatype)?)
         };
+        let base_datatype = info.get_data_type(&base)?;
+
+        if is_zero_literal(&number, &number_datatype)?
+            || is_zero_literal(&base, &base_datatype)?
+        {
+            return Ok(ExprSimplifyResult::Original(original_log_args(
+                num_args, &base, &number,
+            )?));
+        }
+
+        let base_is_valid_literal = is_valid_log_base_literal(&base)?;
 
         match number {
             Expr::Literal(value, _)
-                if value == ScalarValue::new_one(&number_datatype)? =>
+                if value == ScalarValue::new_one(&number_datatype)?
+                    && base_is_valid_literal =>
             {
                 Ok(ExprSimplifyResult::Simplified(lit(ScalarValue::new_zero(
                     &info.get_data_type(&base)?,
                 )?)))
             }
             Expr::ScalarFunction(ScalarFunction { func, mut args })
-                if is_pow(&func) && args.len() == 2 && base == args[0] =>
+                if is_pow(&func)
+                    && args.len() == 2
+                    && base == args[0]
+                    && base_is_valid_literal =>

Review Comment:
   Thanks for tightening the rewrite conditions. I think there is still one edge case here.
   
   The `log(a, power(a, b)) => b` rewrite is still applied when `a` is a valid literal base. That removes the runtime `log` evaluation, which also removes the zero-value validation.
   
   For example:
   
   ```sql
   select log(10.0, power(10.0, column1))
   from (values (-400.0), (2.0)) as t(column1);
   ```
   
   With this rewrite, the result becomes `-400.0, 2.0`.
   
   Without the rewrite, `power(10.0, -400.0)` underflows to `0.0`, so the first row reaches `invoke_with_args` and raises `cannot take logarithm of zero`, which matches the behavior of:
   
   ```sql
   select log(10.0, power(10.0, -400.0));
   ```
   
   Could we either avoid this rewrite unless the value can be proven non-zero, or preserve the runtime zero-value validation when the value side is still an expression? A regression test covering an exponent that underflows to zero would also help lock in the expected behavior.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Fix log(0.0::float8) should error, not return -inf [datafusion]

Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew commented on code in PR #22564:
URL: https://github.com/apache/datafusion/pull/22564#discussion_r3338184839


##########
datafusion/functions/src/math/log.rs:
##########
@@ -358,43 +386,83 @@ impl ScalarUDFImpl for LogFunc {
         } else {
             lit(ScalarValue::new_ten(&number_datatype)?)
         };
+        let base_datatype = info.get_data_type(&base)?;
+
+        if is_zero_literal(&number, &number_datatype)?
+            || is_zero_literal(&base, &base_datatype)?
+        {
+            return Ok(ExprSimplifyResult::Original(original_log_args(
+                num_args, &base, &number,
+            )?));
+        }
+
+        let base_is_valid_literal = is_valid_log_base_literal(&base)?;
 
         match number {
             Expr::Literal(value, _)
-                if value == ScalarValue::new_one(&number_datatype)? =>
+                if value == ScalarValue::new_one(&number_datatype)?
+                    && base_is_valid_literal =>
             {
                 Ok(ExprSimplifyResult::Simplified(lit(ScalarValue::new_zero(
                     &info.get_data_type(&base)?,
                 )?)))
             }
             Expr::ScalarFunction(ScalarFunction { func, mut args })
-                if is_pow(&func) && args.len() == 2 && base == args[0] =>
+                if is_pow(&func)
+                    && args.len() == 2
+                    && base == args[0]
+                    && base_is_valid_literal =>

Review Comment:
   I think we should temporarily abandon these reverse-op simplifications. We do not know
    │ the true runtime range of columns or arbitrary expressions, and the rewrite can
    │ remove validation that should happen at execution time. Safer to only simplify
    │ literal/fully constant cases, or skip the rewrite unless non-zero can be proven.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org