You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/12/02 07:56:45 UTC
[GitHub] [spark] beliefer opened a new pull request #34554: [SPARK-37286][SQL] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
beliefer opened a new pull request #34554:
URL: https://github.com/apache/spark/pull/34554
### What changes were proposed in this pull request?
Currently, the method `compileFilter` and `compileAggregates` is a member of `JDBCRDD`. But it is not reasonable, because the JDBC source knowns how to compile filter and aggregate expressions to itself's dialect well.
### Why are the changes needed?
JDBC source knowns how to compile filter and aggregate expressions to itself's dialect well.
After this PR, we can extend the pushdown(e.g. filter, aggregate) based on different dialect between different JDBC database.
There are two situations:
First, database A and B implement a different number of aggregate functions that meet the SQL standard.
Second, some database implement some aggregate functions that not meet the SQL standard.
### Does this PR introduce _any_ user-facing change?
'No'. Just change the inner implementation.
### How was this patch tested?
Jenkins tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34554:
URL: https://github.com/apache/spark/pull/34554#discussion_r747991324
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala
##########
@@ -193,6 +195,85 @@ abstract class JdbcDialect extends Serializable with Logging{
case _ => value
}
+ /**
+ * Turns a single Filter into a String representing a SQL expression.
+ * Returns None for an unhandled filter.
+ */
+ @Since("3.3.0")
+ def compileFilter(f: Filter): Option[String] = {
Review comment:
The problem is, is there any database that has a different filter/aggregate syntax? The main point of having it in `JdbcDialect` is to allow certain databases to overwrite it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966426522
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145109/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984450209
**[Test build #145853 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145853/testReport)** for PR 34554 at commit [`266fc3c`](https://github.com/apache/spark/commit/266fc3c866c1eb2cc55b07489ed9d1542a9a3d55).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984412307
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50326/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #34554:
URL: https://github.com/apache/spark/pull/34554
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985168212
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50344/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985232094
ping @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966285887
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49578/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966172641
**[Test build #145105 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145105/testReport)** for PR 34554 at commit [`3c238b4`](https://github.com/apache/spark/commit/3c238b413785d8f8760d5f4a48c849fbfeee581f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34554: [SPARK-37286][SQL] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34554:
URL: https://github.com/apache/spark/pull/34554#discussion_r749391888
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala
##########
@@ -193,6 +195,85 @@ abstract class JdbcDialect extends Serializable with Logging{
case _ => value
}
+ /**
+ * Turns a single Filter into a String representing a SQL expression.
+ * Returns None for an unhandled filter.
+ */
+ @Since("3.3.0")
+ def compileFilter(f: Filter): Option[String] = {
Review comment:
This PR adds two new developer APIs in `JdbcDialect` and we need to justify it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #34554: [SPARK-37286][SQL] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #34554:
URL: https://github.com/apache/spark/pull/34554#discussion_r749478053
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala
##########
@@ -193,6 +195,85 @@ abstract class JdbcDialect extends Serializable with Logging{
case _ => value
}
+ /**
+ * Turns a single Filter into a String representing a SQL expression.
+ * Returns None for an unhandled filter.
+ */
+ @Since("3.3.0")
+ def compileFilter(f: Filter): Option[String] = {
Review comment:
There are two situations:
First, database A and B implement a different number of aggregate functions that meet the SQL standard.
Second, some database implement some aggregate functions that not meet the SQL standard.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985446087
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145887/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985508046
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985547064
@cloud-fan Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985282298
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50362/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966330369
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49578/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966288008
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49574/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966626735
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #34554: [SPARK-37286][SQL] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-969835723
Now Spark’s pushdown is that as long as there is a function that can’t be pushed down, it won’t be pushed down at all.
On the other hand, spark does not need to implement partial pushdown, because the additional group by caused also has a lot of overhead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966504313
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49584/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966426522
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145109/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966470695
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49585/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34554:
URL: https://github.com/apache/spark/pull/34554#discussion_r761129237
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala
##########
@@ -193,6 +194,36 @@ abstract class JdbcDialect extends Serializable with Logging{
case _ => value
}
+ /**
+ * Converts aggregate function to String representing a SQL expression.
+ * @param aggregates The aggregate functions to be converted.
+ * @return Converted value.
+ */
+ @Since("3.3.0")
+ def compileAggregates(aggregates: Seq[AggregateFunc]): Option[Seq[String]] = {
Review comment:
since this is an API now, shall we make it `def compileAggregates(func:AggregateFunc): Option[String]` so that it's cleaner and easier to implement?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985317047
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50362/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985168212
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50344/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984400857
**[Test build #145851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145851/testReport)** for PR 34554 at commit [`e39dada`](https://github.com/apache/spark/commit/e39dadadf52ec3d653874d0515e512f5691bdc27).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984541609
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50328/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966317398
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49578/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966172641
**[Test build #145105 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145105/testReport)** for PR 34554 at commit [`3c238b4`](https://github.com/apache/spark/commit/3c238b413785d8f8760d5f4a48c849fbfeee581f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966432062
**[Test build #145115 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145115/testReport)** for PR 34554 at commit [`6fb9bc6`](https://github.com/apache/spark/commit/6fb9bc66a5344d8917fcd71046d3ca51d2cdcc83).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966514085
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984450209
**[Test build #145853 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145853/testReport)** for PR 34554 at commit [`266fc3c`](https://github.com/apache/spark/commit/266fc3c866c1eb2cc55b07489ed9d1542a9a3d55).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984659700
**[Test build #145853 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145853/testReport)** for PR 34554 at commit [`266fc3c`](https://github.com/apache/spark/commit/266fc3c866c1eb2cc55b07489ed9d1542a9a3d55).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984678629
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145853/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
HyukjinKwon edited a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985941431
Build didn't pass - the javadoc build failed above https://github.com/beliefer/spark/actions/runs/1534093499
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966396680
**[Test build #145109 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145109/testReport)** for PR 34554 at commit [`6537574`](https://github.com/apache/spark/commit/65375742d2e5926605fd3e71777811fd50ced44f).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966435165
**[Test build #145116 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145116/testReport)** for PR 34554 at commit [`4406bcc`](https://github.com/apache/spark/commit/4406bcc591841ea2243049e20ac8cc7550e959fc).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer closed pull request #34554: [SPARK-37286][SQL] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
beliefer closed pull request #34554:
URL: https://github.com/apache/spark/pull/34554
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966279613
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49574/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34554: [SPARK-37286][SQL] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34554:
URL: https://github.com/apache/spark/pull/34554#discussion_r749514029
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala
##########
@@ -193,6 +195,85 @@ abstract class JdbcDialect extends Serializable with Logging{
case _ => value
}
+ /**
+ * Turns a single Filter into a String representing a SQL expression.
+ * Returns None for an unhandled filter.
+ */
+ @Since("3.3.0")
+ def compileFilter(f: Filter): Option[String] = {
Review comment:
I think spark will/should only push down SQL-standard predicates/agg functions. Do you have plans to change it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966620785
**[Test build #145116 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145116/testReport)** for PR 34554 at commit [`4406bcc`](https://github.com/apache/spark/commit/4406bcc591841ea2243049e20ac8cc7550e959fc).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class JDBCSuite extends QueryTest with BeforeAndAfter with SharedSparkSession `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966618290
**[Test build #145115 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145115/testReport)** for PR 34554 at commit [`6fb9bc6`](https://github.com/apache/spark/commit/6fb9bc66a5344d8917fcd71046d3ca51d2cdcc83).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984408286
**[Test build #145851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145851/testReport)** for PR 34554 at commit [`e39dada`](https://github.com/apache/spark/commit/e39dadadf52ec3d653874d0515e512f5691bdc27).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985228096
**[Test build #145869 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145869/testReport)** for PR 34554 at commit [`04bb435`](https://github.com/apache/spark/commit/04bb4350ded295530bb8b466a6c5f3d08c8a914d).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985941431
Build didn't pass - the javadoc build failed above https://github.com/beliefer/spark/actions/runs/1534093499
I am going to revert this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966827293
The [SQL] tag is missing in the title
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #34554: [SPARK-37286][SQL] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #34554:
URL: https://github.com/apache/spark/pull/34554#discussion_r748719956
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala
##########
@@ -193,6 +195,85 @@ abstract class JdbcDialect extends Serializable with Logging{
case _ => value
}
+ /**
+ * Turns a single Filter into a String representing a SQL expression.
+ * Returns None for an unhandled filter.
+ */
+ @Since("3.3.0")
+ def compileFilter(f: Filter): Option[String] = {
Review comment:
Yes. I think should lets any database to overwrite it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966626735
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #34554: [SPARK-37286][SQL] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #34554:
URL: https://github.com/apache/spark/pull/34554#discussion_r749478053
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala
##########
@@ -193,6 +195,85 @@ abstract class JdbcDialect extends Serializable with Logging{
case _ => value
}
+ /**
+ * Turns a single Filter into a String representing a SQL expression.
+ * Returns None for an unhandled filter.
+ */
+ @Since("3.3.0")
+ def compileFilter(f: Filter): Option[String] = {
Review comment:
First, database A and B implement a different number of aggregate functions that meet the SQL standard.
Second, some database implement some aggregate functions that not meet the SQL standard.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #34554: [SPARK-37286][SQL] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #34554:
URL: https://github.com/apache/spark/pull/34554#discussion_r749843121
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala
##########
@@ -193,6 +195,85 @@ abstract class JdbcDialect extends Serializable with Logging{
case _ => value
}
+ /**
+ * Turns a single Filter into a String representing a SQL expression.
+ * Returns None for an unhandled filter.
+ */
+ @Since("3.3.0")
+ def compileFilter(f: Filter): Option[String] = {
Review comment:
In my current work, if I need to support some new predicates/aggregation functions first, I will submit a PR first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #34554: [SPARK-37286][SQL] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984376002
Based on discussion offline, reopen this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985441073
**[Test build #145887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145887/testReport)** for PR 34554 at commit [`78eb36c`](https://github.com/apache/spark/commit/78eb36cbeb727c4b9d1ee5fbc11ae3120b49f6fd).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985256184
**[Test build #145887 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145887/testReport)** for PR 34554 at commit [`78eb36c`](https://github.com/apache/spark/commit/78eb36cbeb727c4b9d1ee5fbc11ae3120b49f6fd).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966288008
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49574/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966358747
**[Test build #145105 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145105/testReport)** for PR 34554 at commit [`3c238b4`](https://github.com/apache/spark/commit/3c238b413785d8f8760d5f4a48c849fbfeee581f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966500036
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49585/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984412342
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50326/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984408321
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145851/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984541609
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50328/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984490982
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50328/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985117947
**[Test build #145869 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145869/testReport)** for PR 34554 at commit [`04bb435`](https://github.com/apache/spark/commit/04bb4350ded295530bb8b466a6c5f3d08c8a914d).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34554: [SPARK-37286][SQL] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34554:
URL: https://github.com/apache/spark/pull/34554#discussion_r749390901
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala
##########
@@ -193,6 +195,85 @@ abstract class JdbcDialect extends Serializable with Logging{
case _ => value
}
+ /**
+ * Turns a single Filter into a String representing a SQL expression.
+ * Returns None for an unhandled filter.
+ */
+ @Since("3.3.0")
+ def compileFilter(f: Filter): Option[String] = {
Review comment:
Can you give an example? Which database has a different filter/aggregate syntax? This is SQL standard.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966254895
**[Test build #145109 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145109/testReport)** for PR 34554 at commit [`6537574`](https://github.com/apache/spark/commit/65375742d2e5926605fd3e71777811fd50ced44f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966514085
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966432062
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984408321
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145851/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984412342
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50326/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984678629
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145853/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985142595
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50344/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34554:
URL: https://github.com/apache/spark/pull/34554#discussion_r761661923
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala
##########
@@ -72,7 +72,7 @@ case class JDBCScanBuilder(
if (!jdbcOptions.pushDownAggregate) return false
val dialect = JdbcDialects.get(jdbcOptions.url)
- val compiledAgg = JDBCRDD.compileAggregates(aggregation.aggregateExpressions, dialect)
+ val compiledAgg = aggregation.aggregateExpressions.flatMap(dialect.compileAggregate(_))
Review comment:
This is wrong and is different from the previous behavior. We can't push down anything if one of the aggregate function is not supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #34554:
URL: https://github.com/apache/spark/pull/34554#discussion_r761680072
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala
##########
@@ -72,7 +72,7 @@ case class JDBCScanBuilder(
if (!jdbcOptions.pushDownAggregate) return false
val dialect = JdbcDialects.get(jdbcOptions.url)
- val compiledAgg = JDBCRDD.compileAggregates(aggregation.aggregateExpressions, dialect)
+ val compiledAgg = aggregation.aggregateExpressions.flatMap(dialect.compileAggregate(_))
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985256184
**[Test build #145887 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145887/testReport)** for PR 34554 at commit [`78eb36c`](https://github.com/apache/spark/commit/78eb36cbeb727c4b9d1ee5fbc11ae3120b49f6fd).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985317067
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50362/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985317067
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50362/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966244141
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49574/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966254895
**[Test build #145109 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145109/testReport)** for PR 34554 at commit [`6537574`](https://github.com/apache/spark/commit/65375742d2e5926605fd3e71777811fd50ced44f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966330369
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49578/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966378085
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145105/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966378085
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145105/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966463434
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49584/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #34554: [SPARK-37286] Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-966788389
ping @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985168192
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50344/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985117947
**[Test build #145869 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145869/testReport)** for PR 34554 at commit [`04bb435`](https://github.com/apache/spark/commit/04bb4350ded295530bb8b466a6c5f3d08c8a914d).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985229425
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145869/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985229425
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145869/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985446087
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145887/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984524578
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50328/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-984400857
**[Test build #145851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145851/testReport)** for PR 34554 at commit [`e39dada`](https://github.com/apache/spark/commit/e39dadadf52ec3d653874d0515e512f5691bdc27).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34554: [SPARK-37286][SQL] Move compileAggregates from JDBCRDD to JdbcDialect
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34554:
URL: https://github.com/apache/spark/pull/34554#issuecomment-985941709
Okay but it's fixed in https://github.com/apache/spark/commit/f570d01c0d009bb035d3c89d77661a5432f982cb. Thanks @sarutak
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org