[doris] branch branch-1.1-lts updated: [fix](spark load)The where condition does not take effect when spark load loads the file #13804

[yi...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch branch-1.1-lts
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-1.1-lts by this push:
     new d0fc8323bf [fix](spark load)The where condition does not take effect when spark load loads the file #13804
d0fc8323bf is described below

commit d0fc8323bf08d37d0af7d29a71c8f9532fe072cb
Author: jiafeng.zhang <zh...@gmail.com>
AuthorDate: Mon Oct 31 15:01:46 2022 +0800

    [fix](spark load)The where condition does not take effect when spark load loads the file #13804
    
    The where condition does not take effect when spark load loads the file
---
 .../src/main/java/org/apache/doris/load/loadv2/dpp/SparkDpp.java       | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/fe/spark-dpp/src/main/java/org/apache/doris/load/loadv2/dpp/SparkDpp.java b/fe/spark-dpp/src/main/java/org/apache/doris/load/loadv2/dpp/SparkDpp.java
index 5e0fbe2086..763a0e9ee4 100644
--- a/fe/spark-dpp/src/main/java/org/apache/doris/load/loadv2/dpp/SparkDpp.java
+++ b/fe/spark-dpp/src/main/java/org/apache/doris/load/loadv2/dpp/SparkDpp.java
@@ -727,6 +727,9 @@ public final class SparkDpp implements java.io.Serializable {
         );
 
         Dataset<Row> dataframe = spark.createDataFrame(rowRDD, srcSchema);
+        if (!Strings.isNullOrEmpty(fileGroup.where)) {
+            dataframe = dataframe.where(fileGroup.where);
+        }
         return dataframe;
     }
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org