You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "wForget (via GitHub)" <gi...@apache.org> on 2025/03/06 08:01:14 UTC
[I] Overflow happened on: -9223372036854775808 div -1 [datafusion-comet]
wForget opened a new issue, #1477:
URL: https://github.com/apache/datafusion-comet/issues/1477
### Describe the bug
https://github.com/apache/arrow-rs/pull/7159 only avoids overflow of remainder(%), the divide operation still has similar issues as #1412.
In spark, the `/` operation will be converted to double/decimal division, so only integral divide operation of `Long.MinValue div -1` has inconsistent behavior.
### Steps to reproduce
test case:
```
test("COMET-1412: test smallest signed integer value div -1") {
Seq[(String, Any)](
// ("short", Short.MinValue),
// ("int", Int.MinValue),
("long", Long.MinValue)
// ("double", Double.MinValue)
).foreach { case (t, v) =>
withTable("t1") {
sql(s"create table t1(c1 $t, c2 short) using parquet")
sql(s"insert into t1 values($v, -1)")
withSQLConf(CometConf.COMET_ENABLED.key -> "false "){
sql("select c1 div c2, c1 div -1 from t1 order by c1").show()
}
withSQLConf(CometConf.COMET_ENABLED.key -> "true "){
sql("select c1 div c2, c1 div -1 from t1 order by c1").show()
}
}
}
}
```
result of vanilla spark:
```
+--------------------+--------------------+
| (c1 div c2)| (c1 div -1)|
+--------------------+--------------------+
|-9223372036854775808|-9223372036854775808|
+--------------------+--------------------+
```
error on comet:
```
org.apache.comet.CometNativeException: Arithmetic overflow: Overflow happened on: -9223372036854775808 / -1
at org.apache.comet.Native.executePlan(Native Method)
```
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Overflow happened on: `Long.MinValue div -1` [datafusion-comet]
Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget commented on issue #1477:
URL: https://github.com/apache/datafusion-comet/issues/1477#issuecomment-3043375532
> I can actually get a successful output ( in parity with spark) if I remove the `ORDER BY ` clause . However, the query fails when I do add the `ORDER BY` . [@wForget](https://github.com/wForget) not sure if you can try the test without order by and let me know if the behavior is the same for you too ?
Thanks, I will try it later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Overflow happened on: `Long.MinValue div -1` [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on issue #1477:
URL: https://github.com/apache/datafusion-comet/issues/1477#issuecomment-3030246512
I can actually get a successful output ( in parity with spark) if I remove the `ORDER BY ` clause . However, the query fails when I do add the `ORDER BY` . @wForget not sure if you can try the test without order by and let me know if the behavior is the same for you too ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Overflow happened on: `Long.MinValue div -1` [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on issue #1477:
URL: https://github.com/apache/datafusion-comet/issues/1477#issuecomment-3143235269
This is fixed in the above PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Overflow happened on: `Long.MinValue div -1` [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #1477: Overflow happened on: `Long.MinValue div -1`
URL: https://github.com/apache/datafusion-comet/issues/1477
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org