You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by db...@apache.org on 2021/09/21 15:25:36 UTC

[spark] branch master updated: [SPARK-36814][SQL] Make class ColumnarBatch extendable

This is an automated email from the ASF dual-hosted git repository.

dbtsai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 688b95b  [SPARK-36814][SQL] Make class ColumnarBatch extendable
688b95b is described below

commit 688b95b136571fa559f26e6582fc3fc296f9d1bf
Author: Yufei Gu <yu...@apple.com>
AuthorDate: Tue Sep 21 15:24:55 2021 +0000

    [SPARK-36814][SQL] Make class ColumnarBatch extendable
    
    ### What changes were proposed in this pull request?
    Change class ColumnarBatch to a non-final class
    
    ### Why are the changes needed?
    To support better vectorized reading in multiple data source, ColumnarBatch need to be extendable. For example, To support row-level delete(  https://github.com/apache/iceberg/issues/3141) in Iceberg's vectorized read, we need to filter out deleted rows in a batch, which requires ColumnarBatch to be extendable.
    
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    No test needed.
    
    Closes #34054 from flyrain/columnarbatch-extendable.
    
    Authored-by: Yufei Gu <yu...@apple.com>
    Signed-off-by: DB Tsai <d_...@apple.com>
---
 .../src/main/java/org/apache/spark/sql/vectorized/ColumnarBatch.java    | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ColumnarBatch.java b/sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ColumnarBatch.java
index a2feac8..b5c3ed7 100644
--- a/sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ColumnarBatch.java
+++ b/sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ColumnarBatch.java
@@ -31,7 +31,7 @@ import org.apache.spark.unsafe.types.UTF8String;
  * the entire data loading process.
  */
 @Evolving
-public final class ColumnarBatch implements AutoCloseable {
+public class ColumnarBatch implements AutoCloseable {
   private int numRows;
   private final ColumnVector[] columns;
 

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org