You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by gatorsmile <gi...@git.apache.org> on 2018/08/31 07:07:21 UTC
[GitHub] spark pull request #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/22300
[SPARK-25296][SQL][TEST] Create ExplainSuite
## What changes were proposed in this pull request?
Move the output verification of Explain test cases to a new suite ExplainSuite.
## How was this patch tested?
N/A
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark test3200
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22300.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22300
----
commit e623dab7f1c01ca921917c92feef9bdf1552f1ae
Author: Xiao Li <ga...@...>
Date: 2018-08-31T07:03:09Z
create explain suite.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22300
SGTM. can we use this utility function in `StreamSuite`, too?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22300
**[Test build #95522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95522/testReport)** for PR 22300 at commit [`e623dab`](https://github.com/apache/spark/commit/e623dab7f1c01ca921917c92feef9bdf1552f1ae).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22300
@kiszk ping, could you do that? https://github.com/apache/spark/pull/22300#issuecomment-417706754
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22300
@kiszk Could you help move the related test cases in StreamSuite.scala to here?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22300
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2727/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/22300#discussion_r214266047
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/ExplainSuite.scala ---
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql
+
+import org.apache.spark.sql.test.SharedSQLContext
+import org.apache.spark.sql.types.StructType
+
+class ExplainSuite extends QueryTest with SharedSQLContext {
+ import testImplicits._
+
+ /**
+ * Runs the plan and makes sure the plans contains all of the keywords.
+ */
+ private def checkKeywordsExistsInExplain(df: DataFrame, keywords: String*): Unit = {
--- End diff --
Can't we move this helper func into `QueryTest`?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22300
LGTM, this is cleaner!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22300
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22300#discussion_r214396672
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/ExplainSuite.scala ---
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql
+
+import org.apache.spark.sql.test.SharedSQLContext
+import org.apache.spark.sql.types.StructType
+
+class ExplainSuite extends QueryTest with SharedSQLContext {
+ import testImplicits._
+
+ /**
+ * Runs the plan and makes sure the plans contains all of the keywords.
+ */
+ private def checkKeywordsExistsInExplain(df: DataFrame, keywords: String*): Unit = {
--- End diff --
This is EXPLAIN specific verification. In the next release, we need to further enhance EXPLAIN. I think this helper function will be changed a lot.
If this need to be shared with the other test suites, we can move it to `QueryTest`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22300
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22300
**[Test build #95522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95522/testReport)** for PR 22300 at commit [`e623dab`](https://github.com/apache/spark/commit/e623dab7f1c01ca921917c92feef9bdf1552f1ae).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class ExplainSuite extends QueryTest with SharedSQLContext `
* `case class ExplainSingleData(id: Int)`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22300
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22300
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95522/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22300
Thanks! Merged to master.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org