You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/10/24 14:10:29 UTC
[GitHub] [arrow-datafusion] alamb opened a new pull request, #3939: Add optimizer test for simplifying predicates on timestamps
alamb opened a new pull request, #3939:
URL: https://github.com/apache/arrow-datafusion/pull/3939
Add test for https://github.com/apache/arrow-datafusion/issues/3938
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #3939: Add optimizer test for simplifying predicates on timestamps
Posted by GitBox <gi...@apache.org>.
alamb commented on code in PR #3939:
URL: https://github.com/apache/arrow-datafusion/pull/3939#discussion_r1003883808
##########
datafusion/optimizer/tests/integration-test.rs:
##########
@@ -87,11 +88,11 @@ fn case_when_aggregate() -> Result<()> {
#[test]
fn unsigned_target_type() -> Result<()> {
- let sql = "SELECT * FROM test WHERE col_uint32 > 0";
+ let sql = "SELECT col_utf8 FROM test WHERE col_uint32 > 0";
Review Comment:
"lazy" would also be a good description :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] xudong963 commented on a diff in pull request #3939: Add optimizer test for simplifying predicates on timestamps
Posted by GitBox <gi...@apache.org>.
xudong963 commented on code in PR #3939:
URL: https://github.com/apache/arrow-datafusion/pull/3939#discussion_r1003509064
##########
datafusion/optimizer/tests/integration-test.rs:
##########
@@ -87,11 +88,11 @@ fn case_when_aggregate() -> Result<()> {
#[test]
fn unsigned_target_type() -> Result<()> {
- let sql = "SELECT * FROM test WHERE col_uint32 > 0";
+ let sql = "SELECT col_utf8 FROM test WHERE col_uint32 > 0";
Review Comment:
clever 🤣
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] ursabot commented on pull request #3939: Add optimizer test for simplifying predicates on timestamps
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #3939:
URL: https://github.com/apache/arrow-datafusion/pull/3939#issuecomment-1290951112
Benchmark runs are scheduled for baseline = b5c23c2c29f0e8ffc0f7021c78274176cf2cacb0 and contender = e1f866e99c6648fb72319d39c776f37a3c8db1d2. e1f866e99c6648fb72319d39c776f37a3c8db1d2 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on ec2-t3-xlarge-us-east-2] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/047843845e4d4006b0df4ad4fdcfb67d...fd1b70bbe554448180d4c797782826a9/)
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on test-mac-arm] [test-mac-arm](https://conbench.ursa.dev/compare/runs/ec1bb8aec1294db7acc95237ecd59bdd...d1e61501bc8a4550828ae1b98ad9c15c/)
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on ursa-i9-9960x] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/da646817b6ab4b7587dcc8c9040e873a...23025baff8c14264bccdf122ec325992/)
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on ursa-thinkcentre-m75q] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/a2b0dbab3d4d4a87bceb3c0b2e66fb56...fcd03f5cbe954efcbc994ec252479b90/)
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #3939: Add optimizer test for simplifying predicates on timestamps
Posted by GitBox <gi...@apache.org>.
alamb commented on code in PR #3939:
URL: https://github.com/apache/arrow-datafusion/pull/3939#discussion_r1003364471
##########
datafusion/optimizer/tests/integration-test.rs:
##########
@@ -87,11 +88,11 @@ fn case_when_aggregate() -> Result<()> {
#[test]
fn unsigned_target_type() -> Result<()> {
- let sql = "SELECT * FROM test WHERE col_uint32 > 0";
+ let sql = "SELECT col_utf8 FROM test WHERE col_uint32 > 0";
Review Comment:
As I added some columns to the test data I changed these plans so they selected specific columns rather than `SELECT *` so they didn't diff when the schema changed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #3939: Add optimizer test for simplifying predicates on timestamps
Posted by GitBox <gi...@apache.org>.
alamb commented on code in PR #3939:
URL: https://github.com/apache/arrow-datafusion/pull/3939#discussion_r1003365789
##########
datafusion/optimizer/tests/integration-test.rs:
##########
@@ -225,6 +226,38 @@ fn concat_ws_literals() -> Result<()> {
Ok(())
}
+#[test]
+#[ignore]
+// https://github.com/apache/arrow-datafusion/issues/3938
+fn timestamp_nano_ts_none_predicates() -> Result<()> {
+ let sql = "SELECT col_int32
+ FROM test
+ WHERE col_ts_nano_none < (now() - interval '1 hour')";
+ let plan = test_sql(sql)?;
+ // a scan should have the now()... predicate folded to a single
+ // constant and compared to the column without a cast so it can be
+ // pushed down / pruned
+ let expected = "Projection: test.col_int32\n Filter: test.col_ts_nano_utc < TimestampNanosecond(1666612093000000000, Some(\"UTC\"))\
+ \n TableScan: test projection=[col_int32, col_ts_nano_none]";
+ assert_eq!(expected, format!("{:?}", plan));
+ Ok(())
+}
+
+#[test]
+fn timestamp_nano_ts_utc_predicates() -> Result<()> {
+ let sql = "SELECT col_int32
+ FROM test
+ WHERE col_ts_nano_utc < (now() - interval '1 hour')";
+ let plan = test_sql(sql)?;
+ // a scan should have the now()... predicate folded to a single
+ // constant and compared to the column without a cast so it can be
+ // pushed down / pruned
+ let expected = "Projection: test.col_int32\n Filter: test.col_ts_nano_utc < TimestampNanosecond(1666612093000000000, Some(\"UTC\"))\
Review Comment:
this test -- with `UTC` timezone, works well (it is pretty cool to see it doing the right thing)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] xudong963 merged pull request #3939: Add optimizer test for simplifying predicates on timestamps
Posted by GitBox <gi...@apache.org>.
xudong963 merged PR #3939:
URL: https://github.com/apache/arrow-datafusion/pull/3939
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org