You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by sameeragarwal <gi...@git.apache.org> on 2016/03/16 02:10:06 UTC
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
GitHub user sameeragarwal opened a pull request:
https://github.com/apache/spark/pull/11749
[SPARK-13922][SQL] Filter rows with null attributes in parquet vectorized reader
## What changes were proposed in this pull request?
It's common for many SQL operators to not care about reading `null` values for correctness. Currently, this is achieved by performing `isNotNull` checks (for all relevant columns) on a per-row basis. Pushing these null filters in parquet vectorized reader should bring considerable benefits (especially for cases when the underlying data doesn't contain any nulls or contains all nulls).
## How was this patch tested?
=======================
Fraction of NULLs: 0
=======================
Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
String with Nulls Scan: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------
SQL Parquet Vectorized 1164 / 1333 9.0 111.0 1.0X
PR Vectorized 809 / 882 13.0 77.1 1.4X
PR Vectorized (Null Filtering) 723 / 800 14.5 69.0 1.6X
=======================
Fraction of NULLs: 0.5
=======================
Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
String with Nulls Scan: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------
SQL Parquet Vectorized 983 / 1001 10.7 93.8 1.0X
PR Vectorized 699 / 728 15.0 66.7 1.4X
PR Vectorized (Null Filtering) 722 / 746 14.5 68.9 1.4X
=======================
Fraction of NULLs: 0.95
=======================
Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
String with Nulls Scan: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------
SQL Parquet Vectorized 332 / 343 31.6 31.6 1.0X
PR Vectorized 177 / 180 59.1 16.9 1.9X
PR Vectorized (Null Filtering) 168 / 175 62.4 16.0 2.0X
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sameeragarwal/spark perf-testing
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11749.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11749
----
commit adf9f935d135e6c7f4e94d8a17707577d0cc86f5
Author: Sameer Agarwal <sa...@databricks.com>
Date: 2016-03-11T00:26:18Z
Filter null columns in ColumnarBatch
commit af217fe1a5dc4dbdf015e850de67c2b707f7e541
Author: Sameer Agarwal <sa...@databricks.com>
Date: 2016-03-15T06:29:47Z
Parquet Read Benchmark
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197123489
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53253/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by sameeragarwal <gi...@git.apache.org>.
Github user sameeragarwal commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197504528
thanks, all comments addressed!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on a diff in the pull request:
https://github.com/apache/spark/pull/11749#discussion_r56277661
--- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java ---
@@ -58,6 +58,9 @@
// True if the row is filtered.
private final boolean[] filteredRows;
+ // Column indices that cannot have null values.
+ private final Vector<Integer> nullFilteredColumns;
--- End diff --
change this to a set. I think it's more accurate and makes this easier to use.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197595389
**[Test build #53354 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53354/consoleFull)** for PR 11749 at commit [`0688cf8`](https://github.com/apache/spark/commit/0688cf84958552132aaa8ada960b9c4880b437e6).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by sameeragarwal <gi...@git.apache.org>.
Github user sameeragarwal commented on a diff in the pull request:
https://github.com/apache/spark/pull/11749#discussion_r56417640
--- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java ---
@@ -345,15 +358,25 @@ public void setColumn(int ordinal, ColumnVector column) {
* in this batch will not include this row.
*/
public final void markFiltered(int rowId) {
- assert(filteredRows[rowId] == false);
+ assert(!filteredRows[rowId]);
filteredRows[rowId] = true;
++numRowsFiltered;
}
+ /**
+ * Marks a given column as non-nullable. Any row that has a NULL value for the corresponding
+ * attribute is filtered out.
+ */
+ public final void filterNullsInColumn(int ordinal) {
+ assert(!nullFilteredColumns.contains(ordinal));
--- End diff --
yes, with the set it's not longer required. Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on a diff in the pull request:
https://github.com/apache/spark/pull/11749#discussion_r56277494
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadBenchmark.scala ---
@@ -299,10 +299,112 @@ object ParquetReadBenchmark {
}
}
+ def stringWithNullsScanBenchmark(values: Int, fractionOfNulls: Double): Unit = {
+ withTempPath { dir =>
+ withTempTable("t1", "tempTable") {
+ sqlContext.range(values).registerTempTable("t1")
+ sqlContext.sql(s"select IF(rand(1) < $fractionOfNulls, NULL, cast(id as STRING)) as c1, " +
+ s"IF(rand(2) < $fractionOfNulls, NULL, cast(id as STRING)) as c2 from t1")
+ .write.parquet(dir.getCanonicalPath)
+ sqlContext.read.parquet(dir.getCanonicalPath).registerTempTable("tempTable")
+
+ val benchmark = new Benchmark("String with Nulls Scan", values)
+
+ benchmark.addCase("SQL Parquet Vectorized") { iter =>
+ sqlContext.sql("select sum(length(c2)) from tempTable where c1 is " +
+ "not NULL and c2 is not NULL").collect()
+ }
+
+ val files = SpecificParquetRecordReaderBase.listDirectory(dir).toArray
+ benchmark.addCase("PR Vectorized") { num =>
+ var sum = 0
+ files.map(_.asInstanceOf[String]).foreach { p =>
+ val reader = new UnsafeRowParquetRecordReader
+ try {
+ reader.initialize(p, ("c1" :: "c2" :: Nil).asJava)
+ val batch = reader.resultBatch()
+ while (reader.nextBatch()) {
+ val rowIterator = batch.rowIterator()
+ while (rowIterator.hasNext) {
+ val row = rowIterator.next()
+ val value = row.getUTF8String(0)
+ if (!row.isNullAt(0) && !row.isNullAt(1)) sum += value.numBytes()
+ }
+ }
+ } finally {
+ reader.close()
+ }
+ }
+ }
+
+ benchmark.addCase("PR Vectorized (Null Filtering)") { num =>
+ var sum = 0L
+ files.map(_.asInstanceOf[String]).foreach { p =>
+ val reader = new UnsafeRowParquetRecordReader
+ try {
+ reader.initialize(p, ("c1" :: "c2" :: Nil).asJava)
+ val batch = reader.resultBatch()
+ batch.filterNullsInColumn(0)
+ batch.filterNullsInColumn(1)
+ while (reader.nextBatch()) {
+ val rowIterator = batch.rowIterator()
+ while (rowIterator.hasNext) {
+ sum += rowIterator.next().getUTF8String(0).numBytes()
+ }
+ }
+ } finally {
+ reader.close()
+ }
+ }
+ }
+
+ /*
+ =======================
+ Fraction of NULLs: 0
+ =======================
+
+ Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
+ String with Nulls Scan: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
+ -------------------------------------------------------------------------------------------
+ SQL Parquet Vectorized 1164 / 1333 9.0 111.0 1.0X
+ PR Vectorized 809 / 882 13.0 77.1 1.4X
+ PR Vectorized (Null Filtering) 723 / 800 14.5 69.0 1.6X
+
+ =======================
+ Fraction of NULLs: 0.5
+ =======================
+
+ Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
+ String with Nulls Scan: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
+ -------------------------------------------------------------------------------------------
+ SQL Parquet Vectorized 983 / 1001 10.7 93.8 1.0X
+ PR Vectorized 699 / 728 15.0 66.7 1.4X
+ PR Vectorized (Null Filtering) 722 / 746 14.5 68.9 1.4X
+
+ =======================
--- End diff --
Instead of commenting it this way, can you put the fraction in the benchmark name?
e.g. String with Nulls Scan (95%)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197123483
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197556324
**[Test build #53354 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53354/consoleFull)** for PR 11749 at commit [`0688cf8`](https://github.com/apache/spark/commit/0688cf84958552132aaa8ada960b9c4880b437e6).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197135193
Can you add some test cases to columnarbatchsuite that exercises this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197550621
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on a diff in the pull request:
https://github.com/apache/spark/pull/11749#discussion_r56416653
--- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java ---
@@ -345,15 +358,25 @@ public void setColumn(int ordinal, ColumnVector column) {
* in this batch will not include this row.
*/
public final void markFiltered(int rowId) {
- assert(filteredRows[rowId] == false);
+ assert(!filteredRows[rowId]);
filteredRows[rowId] = true;
++numRowsFiltered;
}
+ /**
+ * Marks a given column as non-nullable. Any row that has a NULL value for the corresponding
+ * attribute is filtered out.
+ */
+ public final void filterNullsInColumn(int ordinal) {
+ assert(!nullFilteredColumns.contains(ordinal));
--- End diff --
I don't think this assert is necessary. I think this is perfectly valid and makes this api easier to use.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197505322
**[Test build #53338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53338/consoleFull)** for PR 11749 at commit [`2d1066f`](https://github.com/apache/spark/commit/2d1066fc853d7d7a9b9ed84af129ab17168f0753).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by sameeragarwal <gi...@git.apache.org>.
Github user sameeragarwal commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197093496
cc @nongli
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on a diff in the pull request:
https://github.com/apache/spark/pull/11749#discussion_r56277607
--- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java ---
@@ -284,11 +287,23 @@ public void reset() {
}
/**
- * Sets the number of rows that are valid.
+ * Sets the number of rows that are valid. Additionally, marks all rows as "filtered" if one or
+ * more of their attributes are part of a non-nullable column.
*/
public void setNumRows(int numRows) {
- assert(numRows <= this.capacity);
+ assert (numRows <= this.capacity);
--- End diff --
did you mean to add a space?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197595610
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53354/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197122998
**[Test build #53253 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53253/consoleFull)** for PR 11749 at commit [`af217fe`](https://github.com/apache/spark/commit/af217fe1a5dc4dbdf015e850de67c2b707f7e541).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197093877
**[Test build #53253 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53253/consoleFull)** for PR 11749 at commit [`af217fe`](https://github.com/apache/spark/commit/af217fe1a5dc4dbdf015e850de67c2b707f7e541).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197550184
**[Test build #53338 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53338/consoleFull)** for PR 11749 at commit [`2d1066f`](https://github.com/apache/spark/commit/2d1066fc853d7d7a9b9ed84af129ab17168f0753).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/11749
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197595608
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197550624
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53338/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...
Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on the pull request:
https://github.com/apache/spark/pull/11749#issuecomment-197553112
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org