You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by sameeragarwal <gi...@git.apache.org> on 2016/03/16 02:10:06 UTC

[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

GitHub user sameeragarwal opened a pull request:

    https://github.com/apache/spark/pull/11749

    [SPARK-13922][SQL] Filter rows with null attributes in parquet vectorized reader

    ## What changes were proposed in this pull request?
    
    It's common for many SQL operators to not care about reading `null` values for correctness. Currently, this is achieved by performing `isNotNull` checks (for all relevant columns) on a per-row basis. Pushing these null filters in parquet vectorized reader should bring considerable benefits (especially for cases when the underlying data doesn't contain any nulls or contains all nulls).
    
    ## How was this patch tested?
    
            =======================
            Fraction of NULLs: 0
            =======================
    
            Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
            String with Nulls Scan:             Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
            -------------------------------------------------------------------------------------------
            SQL Parquet Vectorized                   1164 / 1333          9.0         111.0       1.0X
            PR Vectorized                             809 /  882         13.0          77.1       1.4X
            PR Vectorized (Null Filtering)            723 /  800         14.5          69.0       1.6X
    
            =======================
            Fraction of NULLs: 0.5
            =======================
    
            Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
            String with Nulls Scan:             Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
            -------------------------------------------------------------------------------------------
            SQL Parquet Vectorized                    983 / 1001         10.7          93.8       1.0X
            PR Vectorized                             699 /  728         15.0          66.7       1.4X
            PR Vectorized (Null Filtering)            722 /  746         14.5          68.9       1.4X
    
            =======================
            Fraction of NULLs: 0.95
            =======================
    
            Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
            String with Nulls Scan:             Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
            -------------------------------------------------------------------------------------------
            SQL Parquet Vectorized                    332 /  343         31.6          31.6       1.0X
            PR Vectorized                             177 /  180         59.1          16.9       1.9X
            PR Vectorized (Null Filtering)            168 /  175         62.4          16.0       2.0X

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/sameeragarwal/spark perf-testing

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/11749.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #11749
    
----
commit adf9f935d135e6c7f4e94d8a17707577d0cc86f5
Author: Sameer Agarwal <sa...@databricks.com>
Date:   2016-03-11T00:26:18Z

    Filter null columns in ColumnarBatch

commit af217fe1a5dc4dbdf015e850de67c2b707f7e541
Author: Sameer Agarwal <sa...@databricks.com>
Date:   2016-03-15T06:29:47Z

    Parquet Read Benchmark

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197123489
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53253/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by sameeragarwal <gi...@git.apache.org>.
Github user sameeragarwal commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197504528
  
    thanks, all comments addressed!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11749#discussion_r56277661
  
    --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java ---
    @@ -58,6 +58,9 @@
       // True if the row is filtered.
       private final boolean[] filteredRows;
     
    +  // Column indices that cannot have null values.
    +  private final Vector<Integer> nullFilteredColumns;
    --- End diff --
    
    change this to a set. I think it's more accurate and makes this easier to use.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197595389
  
    **[Test build #53354 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53354/consoleFull)** for PR 11749 at commit [`0688cf8`](https://github.com/apache/spark/commit/0688cf84958552132aaa8ada960b9c4880b437e6).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by sameeragarwal <gi...@git.apache.org>.
Github user sameeragarwal commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11749#discussion_r56417640
  
    --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java ---
    @@ -345,15 +358,25 @@ public void setColumn(int ordinal, ColumnVector column) {
        * in this batch will not include this row.
        */
       public final void markFiltered(int rowId) {
    -    assert(filteredRows[rowId] == false);
    +    assert(!filteredRows[rowId]);
         filteredRows[rowId] = true;
         ++numRowsFiltered;
       }
     
    +  /**
    +   * Marks a given column as non-nullable. Any row that has a NULL value for the corresponding
    +   * attribute is filtered out.
    +   */
    +  public final void filterNullsInColumn(int ordinal) {
    +    assert(!nullFilteredColumns.contains(ordinal));
    --- End diff --
    
    yes, with the set it's not longer required. Fixed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11749#discussion_r56277494
  
    --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadBenchmark.scala ---
    @@ -299,10 +299,112 @@ object ParquetReadBenchmark {
         }
       }
     
    +  def stringWithNullsScanBenchmark(values: Int, fractionOfNulls: Double): Unit = {
    +    withTempPath { dir =>
    +      withTempTable("t1", "tempTable") {
    +        sqlContext.range(values).registerTempTable("t1")
    +        sqlContext.sql(s"select IF(rand(1) < $fractionOfNulls, NULL, cast(id as STRING)) as c1, " +
    +          s"IF(rand(2) < $fractionOfNulls, NULL, cast(id as STRING)) as c2 from t1")
    +          .write.parquet(dir.getCanonicalPath)
    +        sqlContext.read.parquet(dir.getCanonicalPath).registerTempTable("tempTable")
    +
    +        val benchmark = new Benchmark("String with Nulls Scan", values)
    +
    +        benchmark.addCase("SQL Parquet Vectorized") { iter =>
    +          sqlContext.sql("select sum(length(c2)) from tempTable where c1 is " +
    +            "not NULL and c2 is not NULL").collect()
    +        }
    +
    +        val files = SpecificParquetRecordReaderBase.listDirectory(dir).toArray
    +        benchmark.addCase("PR Vectorized") { num =>
    +          var sum = 0
    +          files.map(_.asInstanceOf[String]).foreach { p =>
    +            val reader = new UnsafeRowParquetRecordReader
    +            try {
    +              reader.initialize(p, ("c1" :: "c2" :: Nil).asJava)
    +              val batch = reader.resultBatch()
    +              while (reader.nextBatch()) {
    +                val rowIterator = batch.rowIterator()
    +                while (rowIterator.hasNext) {
    +                  val row = rowIterator.next()
    +                  val value = row.getUTF8String(0)
    +                  if (!row.isNullAt(0) && !row.isNullAt(1)) sum += value.numBytes()
    +                }
    +              }
    +            } finally {
    +              reader.close()
    +            }
    +          }
    +        }
    +
    +        benchmark.addCase("PR Vectorized (Null Filtering)") { num =>
    +          var sum = 0L
    +          files.map(_.asInstanceOf[String]).foreach { p =>
    +            val reader = new UnsafeRowParquetRecordReader
    +            try {
    +              reader.initialize(p, ("c1" :: "c2" :: Nil).asJava)
    +              val batch = reader.resultBatch()
    +              batch.filterNullsInColumn(0)
    +              batch.filterNullsInColumn(1)
    +              while (reader.nextBatch()) {
    +                val rowIterator = batch.rowIterator()
    +                while (rowIterator.hasNext) {
    +                  sum += rowIterator.next().getUTF8String(0).numBytes()
    +                }
    +              }
    +            } finally {
    +              reader.close()
    +            }
    +          }
    +        }
    +
    +        /*
    +        =======================
    +        Fraction of NULLs: 0
    +        =======================
    +
    +        Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
    +        String with Nulls Scan:             Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
    +        -------------------------------------------------------------------------------------------
    +        SQL Parquet Vectorized                   1164 / 1333          9.0         111.0       1.0X
    +        PR Vectorized                             809 /  882         13.0          77.1       1.4X
    +        PR Vectorized (Null Filtering)            723 /  800         14.5          69.0       1.6X
    +
    +        =======================
    +        Fraction of NULLs: 0.5
    +        =======================
    +
    +        Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
    +        String with Nulls Scan:             Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
    +        -------------------------------------------------------------------------------------------
    +        SQL Parquet Vectorized                    983 / 1001         10.7          93.8       1.0X
    +        PR Vectorized                             699 /  728         15.0          66.7       1.4X
    +        PR Vectorized (Null Filtering)            722 /  746         14.5          68.9       1.4X
    +
    +        =======================
    --- End diff --
    
    Instead of commenting it this way, can you put the fraction in the benchmark name?
    
    e.g. String with Nulls Scan (95%)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197123483
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197556324
  
    **[Test build #53354 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53354/consoleFull)** for PR 11749 at commit [`0688cf8`](https://github.com/apache/spark/commit/0688cf84958552132aaa8ada960b9c4880b437e6).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197135193
  
    Can you add some test cases to columnarbatchsuite that exercises this?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197550621
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11749#discussion_r56416653
  
    --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java ---
    @@ -345,15 +358,25 @@ public void setColumn(int ordinal, ColumnVector column) {
        * in this batch will not include this row.
        */
       public final void markFiltered(int rowId) {
    -    assert(filteredRows[rowId] == false);
    +    assert(!filteredRows[rowId]);
         filteredRows[rowId] = true;
         ++numRowsFiltered;
       }
     
    +  /**
    +   * Marks a given column as non-nullable. Any row that has a NULL value for the corresponding
    +   * attribute is filtered out.
    +   */
    +  public final void filterNullsInColumn(int ordinal) {
    +    assert(!nullFilteredColumns.contains(ordinal));
    --- End diff --
    
    I don't think this assert is necessary. I think this is perfectly valid and makes this api easier to use.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197505322
  
    **[Test build #53338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53338/consoleFull)** for PR 11749 at commit [`2d1066f`](https://github.com/apache/spark/commit/2d1066fc853d7d7a9b9ed84af129ab17168f0753).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by sameeragarwal <gi...@git.apache.org>.
Github user sameeragarwal commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197093496
  
    cc @nongli 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11749#discussion_r56277607
  
    --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java ---
    @@ -284,11 +287,23 @@ public void reset() {
       }
     
       /**
    -   * Sets the number of rows that are valid.
    +   * Sets the number of rows that are valid. Additionally, marks all rows as "filtered" if one or
    +   * more of their attributes are part of a non-nullable column.
        */
       public void setNumRows(int numRows) {
    -    assert(numRows <= this.capacity);
    +    assert (numRows <= this.capacity);
    --- End diff --
    
    did you mean to add a space?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197595610
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53354/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197122998
  
    **[Test build #53253 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53253/consoleFull)** for PR 11749 at commit [`af217fe`](https://github.com/apache/spark/commit/af217fe1a5dc4dbdf015e850de67c2b707f7e541).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197093877
  
    **[Test build #53253 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53253/consoleFull)** for PR 11749 at commit [`af217fe`](https://github.com/apache/spark/commit/af217fe1a5dc4dbdf015e850de67c2b707f7e541).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197550184
  
    **[Test build #53338 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53338/consoleFull)** for PR 11749 at commit [`2d1066f`](https://github.com/apache/spark/commit/2d1066fc853d7d7a9b9ed84af129ab17168f0753).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/11749


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197595608
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197550624
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53338/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on the pull request:

    https://github.com/apache/spark/pull/11749#issuecomment-197553112
  
    LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org