You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by mo...@apache.org on 2023/06/12 16:08:05 UTC
[doris] branch master updated: [fix](routine-load) fix stackoverflow bug in routine load (#20704)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 412ca9059e [fix](routine-load) fix stackoverflow bug in routine load (#20704)
412ca9059e is described below
commit 412ca9059ed505661c0ec39880a055ed23c3027e
Author: Mingyu Chen <mo...@163.com>
AuthorDate: Tue Jun 13 00:07:56 2023 +0800
[fix](routine-load) fix stackoverflow bug in routine load (#20704)
When executing routine load job, there may encounter StackOverflowException.
This is because the expr in column setting list will be analyze for each routine load sub task,
and there is a self-reference bug that may cause endless loop when analyzing expr.
The following columns expr list may trigger this bug:
```
columns(col1, col2,
col2=null_or_empty(col2),
col1=null_or_empty(col2))
```
This fix is verified by user, but I can't add regression test for this case, because I can't submit a routine load job
in our regression test, and this bug can only be triggered in routine load.
---
.../java/org/apache/doris/analysis/DataDescription.java | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/DataDescription.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/DataDescription.java
index fadb889921..04f5042d72 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/DataDescription.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/DataDescription.java
@@ -794,13 +794,16 @@ public class DataDescription implements InsertStmt.DataDesc {
// hadoop load only supports the FunctionCallExpr
Expr child1 = predicate.getChild(1);
if (isHadoopLoad && !(child1 instanceof FunctionCallExpr)) {
- throw new AnalysisException("Hadoop load only supports the designated function. "
- + "The error mapping function is:" + child1.toSql());
+ throw new AnalysisException(
+ "Hadoop load only supports the designated function. " + "The error mapping function is:"
+ + child1.toSql());
}
- ImportColumnDesc importColumnDesc = new ImportColumnDesc(column, child1);
+ // Must clone the expr, because in routine load, the expr will be analyzed for each task.
+ Expr cloned = child1.clone();
+ ImportColumnDesc importColumnDesc = new ImportColumnDesc(column, cloned);
parsedColumnExprList.add(importColumnDesc);
- if (child1 instanceof FunctionCallExpr) {
- analyzeColumnToHadoopFunction(column, child1);
+ if (cloned instanceof FunctionCallExpr) {
+ analyzeColumnToHadoopFunction(column, cloned);
}
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org