You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/02/10 06:12:38 UTC
[GitHub] [spark] wangyum opened a new pull request #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
wangyum opened a new pull request #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518
### What changes were proposed in this pull request?
Constraints should be inferred from inequality attributes, for example:
```sql
create table SPARK_30768_1(c1 int, c2 int);
create table SPARK_30768_2(c1 int, c2 int);
explain select t1.* from SPARK_30768_1 t1 join SPARK_30768_2 t2 on (t1.c1 > t2.c1) where t1.c1 = 3;
== Physical Plan ==
*(3) Project [c1#5, c2#6]
+- BroadcastNestedLoopJoin BuildRight, Inner, (c1#5 > c1#7)
:- *(1) Project [c1#5, c2#6]
: +- *(1) Filter (isnotnull(c1#5) AND (c1#5 = 3))
: +- *(1) ColumnarToRow
: +- FileScan parquet default.spark_30768_1[c1#5,c2#6] Batched: true, DataFilters: [isnotnull(c1#5), (c1#5 = 3)], Format: Parquet, Location: InMemoryFileIndex[file:/Users/yumwang/spark-3.0.0-preview2-bin-hadoop2.7/spark-warehous..., PartitionFilters: [], PushedFilters: [IsNotNull(c1), EqualTo(c1,3)], ReadSchema: struct<c1:int,c2:int>
+- BroadcastExchange IdentityBroadcastMode, [id=#60]
+- *(2) Project [c1#7]
+- *(2) Filter isnotnull(c1#7)
+- *(2) ColumnarToRow
+- FileScan parquet default.spark_30768_2[c1#7] Batched: true, DataFilters: [isnotnull(c1#7)], Format: Parquet, Location: InMemoryFileIndex[file:/Users/yumwang/spark-3.0.0-preview2-bin-hadoop2.7/spark-warehous..., PartitionFilters: [], PushedFilters: [IsNotNull(c1)], ReadSchema: struct<c1:int>
```
After this PR:
```sql
== Physical Plan ==
*(3) Project [c1#218, c2#219]
+- BroadcastNestedLoopJoin BuildRight, Inner, (c1#218 > c1#220)
:- *(1) Project [c1#218, c2#219]
: +- *(1) Filter (isnotnull(c1#218) AND (c1#218 = 3))
: +- *(1) ColumnarToRow
: +- FileScan parquet default.spark_30768_1[c1#218,c2#219] Batched: true, DataFilters: [isnotnull(c1#218), (c1#218 = 3)], Format: Parquet, Location: InMemoryFileIndex[file:/Users/yumwang/opensource/spark/sql/core/spark-warehouse/org.apache.spark...., PartitionFilters: [], PushedFilters: [IsNotNull(c1), EqualTo(c1,3)], ReadSchema: struct<c1:int,c2:int>
+- BroadcastExchange IdentityBroadcastMode, [id=#120]
+- *(2) Project [c1#220]
+- *(2) Filter ((3 > c1#220) AND isnotnull(c1#220))
+- *(2) ColumnarToRow
+- FileScan parquet default.spark_30768_2[c1#220] Batched: true, DataFilters: [(3 > c1#220), isnotnull(c1#220)], Format: Parquet, Location: InMemoryFileIndex[file:/Users/yumwang/opensource/spark/sql/core/spark-warehouse/org.apache.spark...., PartitionFilters: [], PushedFilters: [LessThan(c1,3), IsNotNull(c1)], ReadSchema: struct<c1:int>
```
Hive support this feature:
```sql
hive> explain select t1.* from SPARK_30768_1 t1 join SPARK_30768_2 t2 on (t1.c1 > t2.c1) where t1.c1 = 3;
Warning: Map Join MAPJOIN[13][bigTable=?] in task 'Stage-3:MAPRED' is a cross product
OK
STAGE DEPENDENCIES:
Stage-4 is a root stage
Stage-3 depends on stages: Stage-4
Stage-0 depends on stages: Stage-3
STAGE PLANS:
Stage: Stage-4
Map Reduce Local Work
Alias -> Map Local Tables:
$hdt$_0:t1
Fetch Operator
limit: -1
Alias -> Map Local Operator Tree:
$hdt$_0:t1
TableScan
alias: t1
filterExpr: (c1 = 3) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
predicate: (c1 = 3) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: c2 (type: int)
outputColumnNames: _col1
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
HashTable Sink Operator
keys:
0
1
Stage: Stage-3
Map Reduce
Map Operator Tree:
TableScan
alias: t2
filterExpr: (c1 < 3) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
predicate: (c1 < 3) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0
1
outputColumnNames: _col1
Statistics: Num rows: 1 Data size: 1 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: 3 (type: int), _col1 (type: int)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 1 Basic stats: PARTIAL Column stats: NONE
File Output Operator
compressed: false
Statistics: Num rows: 1 Data size: 1 Basic stats: PARTIAL Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Execution mode: vectorized
Local Work:
Map Reduce Local Work
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
Time taken: 5.491 seconds, Fetched: 71 row(s)
```
### Why are the changes needed?
Improve query performance.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Unit test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584002293
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586930402
**[Test build #118570 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118570/testReport)** for PR 27518 at commit [`f9a90aa`](https://github.com/apache/spark/commit/f9a90aae3545c6ef1f1451cd25f084d2cbbed5ea).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586304986
**[Test build #118434 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118434/testReport)** for PR 27518 at commit [`d4eb2d7`](https://github.com/apache/spark/commit/d4eb2d7ed17016e5c7e5983d8e0ee2411833fb92).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585274328
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23067/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-596089277
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24245/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] peter-toth commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
peter-toth commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585251432
@wangyum, I'm a but confused now. As far as I see based on your changes, in this PR you substitute `attribute = constant` form of constraints into other constraints to infer new constraints.
So this PR will help with your example query as from `select t1.* from SPARK_30768_1 t1 join SPARK_30768_2 t2 on (t1.c1 > t2.c1) where t1.c1 = 3` the new constraint `3 > t2.c1` is inferred.
It will also help with those cases I mentioned:
`select t1.* from SPARK_30768_1 t1 join SPARK_30768_2 t2 on abs(t1.c1) = t2.c1 where t1.c1 = 3` as `abs(3) = t2.c1` will be inferred.
It will also help here: https://github.com/apache/spark/pull/27252#discussion_r378184866 as after both this PR **and** https://github.com/apache/spark/pull/27252 are merged. It will solve the issue in the comment of @cloud-fan:
> If we have cast(a, dt) = b and a = 1, seems we can also infer cast(1, dt) = b
But, I don't see how it will help with the example query here: https://github.com/apache/spark/pull/27518#issuecomment-585189218
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586864420
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118526/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-591478420
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23728/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585273754
**[Test build #118309 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118309/testReport)** for PR 27518 at commit [`999126c`](https://github.com/apache/spark/commit/999126cf899816e4c4f2035f70c9099fceb63516).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
wangyum commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585666369
@peter-toth Please go ahead. I reverted the first commit because this test will fail:
```scala
test("Constraints shouldn't be inferred from cast equality constraint(filter lower data type)") {
val testRelation1 = LocalRelation('a.int)
val testRelation2 = LocalRelation('b.long)
val originalLeft = testRelation1.where('a === 1).subquery('left)
val originalRight = testRelation2.subquery('right)
val left = testRelation1.where(IsNotNull('a) && 'a === 1).subquery('left)
val right = testRelation2.where(IsNotNull('b)).subquery('right)
Seq(Some("left.a".attr.cast(LongType) === "right.b".attr),
Some("right.b".attr === "left.a".attr.cast(LongType))).foreach { condition =>
testConstraintsAfterJoin(originalLeft, originalRight, left, right, Inner, condition)
}
}
```
```
== FAIL: Plans do not match ===
'Join Inner, (b#0L = cast(a#0 as bigint)) 'Join Inner, (b#0L = cast(a#0 as bigint))
!:- Filter (((1 = a#0) AND isnotnull(a#0)) AND (cast(a#0 as bigint) = cast(1 as bigint))) :- Filter ((1 = a#0) AND isnotnull(a#0))
: +- LocalRelation <empty>, [a#0] : +- LocalRelation <empty>, [a#0]
!+- Filter (isnotnull(b#0L) AND (b#0L = cast(1 as bigint))) +- Filter isnotnull(b#0L)
+- LocalRelation <empty>, [b#0L] +- LocalRelation <empty>, [b#0L]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584000761
**[Test build #118120 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118120/testReport)** for PR 27518 at commit [`b59a810`](https://github.com/apache/spark/commit/b59a81075d4b2692e1a68974a452143cb94e2778).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586428286
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-601896971
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120106/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584001635
**[Test build #118131 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118131/testReport)** for PR 27518 at commit [`b59a810`](https://github.com/apache/spark/commit/b59a81075d4b2692e1a68974a452143cb94e2778).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586813204
**[Test build #118526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118526/testReport)** for PR 27518 at commit [`3c9968a`](https://github.com/apache/spark/commit/3c9968a64969e4bcb2baa31bc7833764314ed16d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [SPARK-30768][SQL]
Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-596089133
**[Test build #119516 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119516/testReport)** for PR 27518 at commit [`248e3cc`](https://github.com/apache/spark/commit/248e3cca6c5f77b768ff738ae5fecc1f9476defb).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585274328
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23067/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-583972873
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22886/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-583972538
**[Test build #118120 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118120/testReport)** for PR 27518 at commit [`b59a810`](https://github.com/apache/spark/commit/b59a81075d4b2692e1a68974a452143cb94e2778).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584245017
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118163/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586721547
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118505/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-591601331
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586428302
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118434/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-596112090
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119516/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-590682416
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585274318
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584245017
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118163/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586427537
**[Test build #118434 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118434/testReport)** for PR 27518 at commit [`d4eb2d7`](https://github.com/apache/spark/commit/d4eb2d7ed17016e5c7e5983d8e0ee2411833fb92).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584100238
**[Test build #118131 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118131/testReport)** for PR 27518 at commit [`b59a810`](https://github.com/apache/spark/commit/b59a81075d4b2692e1a68974a452143cb94e2778).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585324024
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584105875
**[Test build #118163 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118163/testReport)** for PR 27518 at commit [`b59a810`](https://github.com/apache/spark/commit/b59a81075d4b2692e1a68974a452143cb94e2778).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-601896971
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120106/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584243990
**[Test build #118163 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118163/testReport)** for PR 27518 at commit [`b59a810`](https://github.com/apache/spark/commit/b59a81075d4b2692e1a68974a452143cb94e2778).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584002293
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
wangyum commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585189218
@peter-toth I'd like to handle another case in this PR:
```scala
spark.sql("CREATE TABLE `dw_user_state_history` (`id` DECIMAL(18,0), `change_time` date, `to_state` DECIMAL(9,0)) USING parquet")
spark.sql("CREATE TABLE `dw_user_cntry_hist` (`user_id` DECIMAL(18,0), `start_dt` DATE, `end_dt` DATE) USING parquet")
spark.sql(
"""
|SELECT
| count(*)
|FROM
| dw_user_state_history ush
| INNER JOIN dw_user_cntry_hist uch ON (uch.user_id = ush.id AND CAST(ush.change_time AS date) >= uch.start_dt AND CAST(ush.change_time AS date) < uch.end_dt)
|WHERE
| change_time between '2019-07-01 00:00:00' AND '2019-07-02 00:00:00'
|""".stripMargin).explain()
```
The exepected physical plan:
```
== Physical Plan ==
*(3) HashAggregate(keys=[], functions=[count(1)])
+- Exchange SinglePartition, true, [id=#139]
+- *(2) HashAggregate(keys=[], functions=[partial_count(1)])
+- *(2) Project
+- *(2) BroadcastHashJoin [id#219], [user_id#222], Inner, BuildRight, ((change_time#220 >= start_dt#223) AND (change_time#220 < end_dt#224))
:- *(2) Project [id#219, change_time#220]
: +- *(2) Filter (((isnotnull(change_time#220) AND (change_time#220 >= 18078)) AND (change_time#220 <= 18079)) AND isnotnull(id#219))
: +- *(2) ColumnarToRow
: +- FileScan parquet default.dw_user_state_history[id#219,change_time#220] Batched: true, DataFilters: [isnotnull(change_time#220), (change_time#220 >= 18078), (change_time#220 <= 18079), isnotnull(id..., Format: Parquet, Location: InMemoryFileIndex[file:/Users/yumwang/opensource/spark/sql/core/spark-warehouse/org.apache.spark...., PartitionFilters: [], PushedFilters: [IsNotNull(change_time), GreaterThanOrEqual(change_time,2019-07-01), LessThanOrEqual(change_time,..., ReadSchema: struct<id:decimal(18,0),change_time:date>
+- BroadcastExchange HashedRelationBroadcastMode(List(input[0, decimal(18,0), true])), [id=#133]
+- *(1) Project [user_id#222, start_dt#223, end_dt#224]
+- *(1) Filter (((((isnotnull(end_dt#224) AND (start_dt#223 <= 18079)) AND isnotnull(user_id#222)) AND (start_dt#223 < end_dt#224)) AND isnotnull(start_dt#223)) AND (end_dt#224 > 18078))
+- *(1) ColumnarToRow
+- FileScan parquet default.dw_user_cntry_hist[user_id#222,start_dt#223,end_dt#224] Batched: true, DataFilters: [isnotnull(end_dt#224), (start_dt#223 <= 18079), isnotnull(user_id#222), (start_dt#223 < end_dt#2..., Format: Parquet, Location: InMemoryFileIndex[file:/Users/yumwang/opensource/spark/sql/core/spark-warehouse/org.apache.spark...., PartitionFilters: [], PushedFilters: [IsNotNull(end_dt), LessThanOrEqual(start_dt,2019-07-02), IsNotNull(user_id), IsNotNull(start_dt)..., ReadSchema: struct<user_id:decimal(18,0),start_dt:date,end_dt:date>
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [SPARK-30768][SQL]
Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-596111919
**[Test build #119516 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119516/testReport)** for PR 27518 at commit [`248e3cc`](https://github.com/apache/spark/commit/248e3cca6c5f77b768ff738ae5fecc1f9476defb).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-596112084
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586717129
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586864420
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118526/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
wangyum commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586929232
Metrics of Analyzer/Optimizer Rules for `TPCDSQuerySuite`
Before this PR:
```scala
02:32:26.475 WARN org.apache.spark.sql.TPCDSQuerySuite:
=== Metrics of Analyzer/Optimizer Rules ===
Total number of runs: 224786
Total time: 45.803692546 seconds
Rule Effective Time / Total Time Effective Runs / Total Runs
org.apache.spark.sql.catalyst.optimizer.Optimizer$OptimizeSubqueries 6898974687 / 8840587642 47 / 772
org.apache.spark.sql.catalyst.optimizer.ColumnPruning 671684614 / 2740648387 327 / 2364
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveSubquery 1637851386 / 1780942834 51 / 2159
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAggregateFunctions 624455301 / 1744459696 49 / 2159
org.apache.spark.sql.catalyst.analysis.DecimalPrecision 1136163267 / 1355664883 361 / 2159
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences 958843179 / 1132184862 813 / 2159
org.apache.spark.sql.catalyst.optimizer.PruneFilters 21041535 / 875480663 5 / 1978
org.apache.spark.sql.catalyst.optimizer.BooleanSimplification 7949392 / 757989983 4 / 1592
org.apache.spark.sql.catalyst.analysis.TypeCoercion$ImplicitTypeCasts 301831588 / 657645402 78 / 2159
org.apache.spark.sql.catalyst.optimizer.PushDownPredicates 374351767 / 641828516 758 / 1979
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveMissingReferences 14541719 / 632536654 10 / 2159
org.apache.spark.sql.catalyst.optimizer.NullPropagation 39937489 / 587603524 42 / 1592
org.apache.spark.sql.catalyst.optimizer.ReorderJoin 244728546 / 572349474 177 / 1592
org.apache.spark.sql.catalyst.optimizer.ConstantFolding 158687531 / 564225840 194 / 1592
org.apache.spark.sql.catalyst.optimizer.ReorderAssociativeOperator 0 / 547165121 0 / 1592
org.apache.spark.sql.catalyst.optimizer.SimplifyConditionals 0 / 532921745 0 / 1592
org.apache.spark.sql.catalyst.optimizer.SimplifyBinaryComparison 0 / 527530671 0 / 1592
org.apache.spark.sql.catalyst.optimizer.RemoveNoopOperators 48706059 / 526075410 116 / 2364
org.apache.spark.sql.catalyst.optimizer.SimplifyExtractValueOps 0 / 511807420 0 / 1592
org.apache.spark.sql.catalyst.optimizer.SimplifyCasts 45722796 / 510124455 83 / 1592
org.apache.spark.sql.catalyst.optimizer.SimplifyCaseConversionExpressions 0 / 493219669 0 / 1592
org.apache.spark.sql.catalyst.optimizer.OptimizeIn 12300150 / 491448204 27 / 1592
org.apache.spark.sql.catalyst.analysis.UpdateAttributeNullability 20208595 / 484413473 12 / 674
org.apache.spark.sql.catalyst.optimizer.RemoveDispensableExpressions 0 / 480755936 0 / 1592
org.apache.spark.sql.catalyst.optimizer.LikeSimplification 799298 / 463609379 1 / 1592
org.apache.spark.sql.catalyst.optimizer.CollapseProject 92682031 / 457556748 215 / 1978
org.apache.spark.sql.catalyst.optimizer.ReplaceNullWithFalseInPredicate 0 / 456658852 0 / 1592
org.apache.spark.sql.catalyst.analysis.TypeCoercion$FunctionArgumentConversion 227217097 / 442233854 56 / 2159
org.apache.spark.sql.catalyst.optimizer.InferFiltersFromConstraints 406969136 / 435459925 278 / 386
```
After this PR:
```scala
02:28:49.937 WARN org.apache.spark.sql.TPCDSQuerySuite:
=== Metrics of Analyzer/Optimizer Rules ===
Total number of runs: 224786
Total time: 47.011460872 seconds
Rule Effective Time / Total Time Effective Runs / Total Runs
org.apache.spark.sql.catalyst.optimizer.Optimizer$OptimizeSubqueries 7073196527 / 8950854926 47 / 772
org.apache.spark.sql.catalyst.optimizer.ColumnPruning 724531405 / 2931435267 327 / 2364
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveSubquery 1789988207 / 1942196179 51 / 2159
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAggregateFunctions 717122415 / 1838219499 49 / 2159
org.apache.spark.sql.catalyst.analysis.DecimalPrecision 1276704718 / 1524939842 361 / 2159
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences 1003010234 / 1192920997 813 / 2159
org.apache.spark.sql.catalyst.optimizer.PruneFilters 21471500 / 952660373 5 / 1978
org.apache.spark.sql.catalyst.optimizer.BooleanSimplification 9761473 / 773205469 4 / 1592
org.apache.spark.sql.catalyst.analysis.TypeCoercion$ImplicitTypeCasts 329430374 / 734067405 78 / 2159
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveMissingReferences 12112700 / 709719203 10 / 2159
org.apache.spark.sql.catalyst.optimizer.PushDownPredicates 399755753 / 663934266 758 / 1979
org.apache.spark.sql.catalyst.optimizer.ReorderJoin 266383835 / 593070778 177 / 1592
org.apache.spark.sql.catalyst.optimizer.NullPropagation 35781183 / 578417823 42 / 1592
org.apache.spark.sql.catalyst.optimizer.RemoveNoopOperators 48387593 / 540807035 116 / 2364
org.apache.spark.sql.catalyst.optimizer.SimplifyBinaryComparison 0 / 535708634 0 / 1592
org.apache.spark.sql.catalyst.optimizer.ConstantFolding 114115625 / 533552246 194 / 1592
org.apache.spark.sql.catalyst.optimizer.OptimizeIn 12632788 / 529487507 27 / 1592
org.apache.spark.sql.catalyst.optimizer.SimplifyCaseConversionExpressions 0 / 513745956 0 / 1592
org.apache.spark.sql.catalyst.optimizer.SimplifyConditionals 0 / 508586160 0 / 1592
org.apache.spark.sql.catalyst.optimizer.InferFiltersFromConstraints 459543325 / 490828096 278 / 386
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584101190
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118131/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-587045960
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118570/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-596089133
**[Test build #119516 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119516/testReport)** for PR 27518 at commit [`248e3cc`](https://github.com/apache/spark/commit/248e3cca6c5f77b768ff738ae5fecc1f9476defb).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586931488
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-596089273
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-591478406
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584106246
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22924/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584105875
**[Test build #118163 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118163/testReport)** for PR 27518 at commit [`b59a810`](https://github.com/apache/spark/commit/b59a81075d4b2692e1a68974a452143cb94e2778).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] peter-toth commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
peter-toth commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585659476
> @peter-toth [999126c](https://github.com/apache/spark/commit/999126cf899816e4c4f2035f70c9099fceb63516) helps the example query.
Thanks @wangyum. I see now that you basically reverted your first commit.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-590723898
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-590723901
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118895/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586930402
**[Test build #118570 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118570/testReport)** for PR 27518 at commit [`f9a90aa`](https://github.com/apache/spark/commit/f9a90aae3545c6ef1f1451cd25f084d2cbbed5ea).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584002305
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22894/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585324024
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586864412
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-587045960
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118570/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586721547
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118505/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586813454
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586721528
**[Test build #118505 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118505/testReport)** for PR 27518 at commit [`df679e6`](https://github.com/apache/spark/commit/df679e673887dd9b8a795c8f5447ba871a6be03f).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-596112090
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119516/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585324040
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118309/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-601896961
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-583972538
**[Test build #118120 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118120/testReport)** for PR 27518 at commit [`b59a810`](https://github.com/apache/spark/commit/b59a81075d4b2692e1a68974a452143cb94e2778).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586306807
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-591477538
**[Test build #118980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118980/testReport)** for PR 27518 at commit [`bfa6039`](https://github.com/apache/spark/commit/bfa60393789673b1cd77fbda038cdc8c17c1a866).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-596089277
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24245/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-587044835
**[Test build #118570 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118570/testReport)** for PR 27518 at commit [`f9a90aa`](https://github.com/apache/spark/commit/f9a90aae3545c6ef1f1451cd25f084d2cbbed5ea).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
wangyum commented on a change in pull request #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#discussion_r377057880
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala
##########
@@ -67,6 +67,10 @@ trait ConstraintHelper {
val candidateConstraints = constraints - eq
Review comment:
I think the method documentation is correct. We support the following pattern before this PR:
```sql
spark-sql> explain select t1.* from SPARK_30768_1 t1 join SPARK_30768_2 t2 on (t1.c1 = t2.c1) where t1.c1 > 3;
== Physical Plan ==
*(2) Project [c1#5, c2#6]
+- *(2) BroadcastHashJoin [c1#5], [c1#7], Inner, BuildRight
:- *(2) Project [c1#5, c2#6]
: +- *(2) Filter (isnotnull(c1#5) AND (c1#5 > 3))
: +- *(2) ColumnarToRow
: +- FileScan parquet default.spark_30768_1[c1#5,c2#6] Batched: true, DataFilters: [isnotnull(c1#5), (c1#5 > 3)], Format: Parquet, Location: InMemoryFileIndex[file:/Users/yumwang/spark-3.0.0-preview2-bin-hadoop2.7/spark-warehous..., PartitionFilters: [], PushedFilters: [IsNotNull(c1), GreaterThan(c1,3)], ReadSchema: struct<c1:int,c2:int>
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint))), [id=#233]
+- *(1) Project [c1#7]
+- *(1) Filter ((c1#7 > 3) AND isnotnull(c1#7))
+- *(1) ColumnarToRow
+- FileScan parquet default.spark_30768_2[c1#7] Batched: true, DataFilters: [(c1#7 > 3), isnotnull(c1#7)], Format: Parquet, Location: InMemoryFileIndex[file:/Users/yumwang/spark-3.0.0-preview2-bin-hadoop2.7/spark-warehous..., PartitionFilters: [], PushedFilters: [GreaterThan(c1,3), IsNotNull(c1)], ReadSchema: struct<c1:int>
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585274318
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-591601331
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586813204
**[Test build #118526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118526/testReport)** for PR 27518 at commit [`3c9968a`](https://github.com/apache/spark/commit/3c9968a64969e4bcb2baa31bc7833764314ed16d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#discussion_r376930451
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala
##########
@@ -67,6 +67,10 @@ trait ConstraintHelper {
val candidateConstraints = constraints - eq
Review comment:
@wangyum, can you update the method documentation above as well?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586813454
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585324040
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118309/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586813459
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23282/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586813459
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23282/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-591601345
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118980/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-601764330
**[Test build #120106 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120106/testReport)** for PR 27518 at commit [`248e3cc`](https://github.com/apache/spark/commit/248e3cca6c5f77b768ff738ae5fecc1f9476defb).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584106235
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586428302
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118434/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [SPARK-30768][SQL]
Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-601764330
**[Test build #120106 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120106/testReport)** for PR 27518 at commit [`248e3cc`](https://github.com/apache/spark/commit/248e3cca6c5f77b768ff738ae5fecc1f9476defb).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
wangyum commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584000956
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
wangyum commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585273507
@peter-toth https://github.com/apache/spark/pull/27518/commits/999126cf899816e4c4f2035f70c9099fceb63516 helps the example query.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [SPARK-30768][SQL]
Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-601896282
**[Test build #120106 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120106/testReport)** for PR 27518 at commit [`248e3cc`](https://github.com/apache/spark/commit/248e3cca6c5f77b768ff738ae5fecc1f9476defb).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584001049
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584001054
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118120/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586306815
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23191/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] peter-toth edited a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
peter-toth edited a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585659476
> @peter-toth [999126c](https://github.com/apache/spark/commit/999126cf899816e4c4f2035f70c9099fceb63516) helps the example query.
Thanks @wangyum. I see now that you basically reverted your first commit.
If you don't mind I would open a PR that does some kind of constant propagation among constraints. Probably a bit similar to the first commit in this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-591478406
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-590683927
**[Test build #118895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118895/testReport)** for PR 27518 at commit [`f9a90aa`](https://github.com/apache/spark/commit/f9a90aae3545c6ef1f1451cd25f084d2cbbed5ea).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584001049
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584245003
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-591600346
**[Test build #118980 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118980/testReport)** for PR 27518 at commit [`bfa6039`](https://github.com/apache/spark/commit/bfa60393789673b1cd77fbda038cdc8c17c1a866).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-591477538
**[Test build #118980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118980/testReport)** for PR 27518 at commit [`bfa6039`](https://github.com/apache/spark/commit/bfa60393789673b1cd77fbda038cdc8c17c1a866).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586306815
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23191/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-590682156
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-590723898
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-596089273
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
wangyum commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584105431
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586864412
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586717130
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23260/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-590682422
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23644/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584106235
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586717827
**[Test build #118505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118505/testReport)** for PR 27518 at commit [`df679e6`](https://github.com/apache/spark/commit/df679e673887dd9b8a795c8f5447ba871a6be03f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-587045950
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585323887
**[Test build #118309 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118309/testReport)** for PR 27518 at commit [`999126c`](https://github.com/apache/spark/commit/999126cf899816e4c4f2035f70c9099fceb63516).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-601896961
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-591478420
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23728/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586721541
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-590683927
**[Test build #118895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118895/testReport)** for PR 27518 at commit [`f9a90aa`](https://github.com/apache/spark/commit/f9a90aae3545c6ef1f1451cd25f084d2cbbed5ea).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-587045950
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584106246
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22924/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584101178
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586931509
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23326/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586717827
**[Test build #118505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118505/testReport)** for PR 27518 at commit [`df679e6`](https://github.com/apache/spark/commit/df679e673887dd9b8a795c8f5447ba871a6be03f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586931509
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23326/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584101190
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118131/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-590723901
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118895/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584002305
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22894/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586717129
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-590723669
**[Test build #118895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118895/testReport)** for PR 27518 at commit [`f9a90aa`](https://github.com/apache/spark/commit/f9a90aae3545c6ef1f1451cd25f084d2cbbed5ea).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-590682416
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586717130
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23260/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-583972870
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584245003
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586306807
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586304986
**[Test build #118434 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118434/testReport)** for PR 27518 at commit [`d4eb2d7`](https://github.com/apache/spark/commit/d4eb2d7ed17016e5c7e5983d8e0ee2411833fb92).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-590682422
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23644/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] peter-toth commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
peter-toth commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584839346
Is the title correct? Isn't what you do here is constant propagation among constraints?
Obviously it helps with inequalities of attributes, but also helps where you have complex equalities like `select t1.* from SPARK_30768_1 t1 join SPARK_30768_2 t2 on f(t1.c1) = t2.c1 where t1.c1 = 3` (where `f` is a deterministic function).
BTW, I have a PR open to enhance `ConstantPropagation`: https://github.com/apache/spark/pull/24553 where among other enhancements I try to extend propagation of attribute => constant mapping to propagation of deterministic expression => constant mapping. Maybe you could do expression => constant propagation here as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584101178
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584001054
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118120/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-584001635
**[Test build #118131 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118131/testReport)** for PR 27518 at commit [`b59a810`](https://github.com/apache/spark/commit/b59a81075d4b2692e1a68974a452143cb94e2778).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-583972870
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[SPARK-30768][SQL] Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-583972873
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22886/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] peter-toth edited a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
peter-toth edited a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585251432
@wangyum, I'm a but confused now. As far as I see based on your changes, in this PR you substitute `attribute = constant` form of constraints into other constraints to infer new constraints.
So this PR will help with your example query as from `select t1.* from SPARK_30768_1 t1 join SPARK_30768_2 t2 on (t1.c1 > t2.c1) where t1.c1 = 3` the new constraint `3 > t2.c1` is inferred.
It will also help with those cases I mentioned:
`select t1.* from SPARK_30768_1 t1 join SPARK_30768_2 t2 on abs(t1.c1) = t2.c1 where t1.c1 = 3` as `abs(3) = t2.c1` will be inferred.
It will also help here: https://github.com/apache/spark/pull/27252#discussion_r378184866 as after both this PR **and** https://github.com/apache/spark/pull/27252 are merged, it will solve the issue in the comment of @cloud-fan:
> If we have cast(a, dt) = b and a = 1, seems we can also infer cast(1, dt) = b
But, I don't see how it will help with the example query here: https://github.com/apache/spark/pull/27518#issuecomment-585189218
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586931488
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586721541
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL]
Constraints should be inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586864086
**[Test build #118526 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118526/testReport)** for PR 27518 at commit [`3c9968a`](https://github.com/apache/spark/commit/3c9968a64969e4bcb2baa31bc7833764314ed16d).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* ` case class WorkerDecommission(`
* ` case class DecommissionExecutor(executorId: String) extends CoarseGrainedClusterMessage`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-586428286
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518: [SPARK-30768][SQL]
Constraints inferred from inequality attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-596112084
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-591601345
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118980/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27518:
[WIP][SPARK-30768][SQL] Constraints should be inferred from inequality
attributes
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27518: [WIP][SPARK-30768][SQL] Constraints should be inferred from inequality attributes
URL: https://github.com/apache/spark/pull/27518#issuecomment-585273754
**[Test build #118309 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118309/testReport)** for PR 27518 at commit [`999126c`](https://github.com/apache/spark/commit/999126cf899816e4c4f2035f70c9099fceb63516).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org