You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/07/23 07:55:17 UTC
[GitHub] [incubator-doris] morningman opened a new pull request #4158: [RoutineLoad] Support modify routine load job
morningman opened a new pull request #4158:
URL: https://github.com/apache/incubator-doris/pull/4158
## Proposed changes
Support ALTER ROUTINE LOAD JOB stmt, for example:
```
alter routine load db1.label1
properties
(
"desired_concurrent_number"="3",
"max_batch_interval" = "5",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200",
"strict_mode" = "false",
"timezone" = "+08:00"
)
```
Details can be found in `alter-routine-load.md`
## Types of changes
- [x] New feature (non-breaking change which adds functionality)
## Checklist
- [x] I have create an issue on fix #4157, and have described the bug/feature there in detail
- [x] Compiling and unit tests pass locally with my changes
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] If this change need a document change, I have updated the document
- [x] Any dependent changes have been merged
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #4158: [RoutineLoad] Support modify routine load job
Posted by GitBox <gi...@apache.org>.
EmmyMiao87 commented on a change in pull request #4158:
URL: https://github.com/apache/incubator-doris/pull/4158#discussion_r464287690
##########
File path: fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java
##########
@@ -1397,4 +1401,36 @@ public void readFields(DataInput in) throws IOException {
throw new IOException("error happens when parsing create routine load stmt: " + origStmt, e);
}
}
+
+ abstract public void modifyProperties(AlterRoutineLoadStmt stmt) throws DdlException;
+
+ abstract public void replayModifyProperties(AlterRoutineLoadJobOperationLog log);
Review comment:
Same as above.
##########
File path: fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java
##########
@@ -1397,4 +1401,36 @@ public void readFields(DataInput in) throws IOException {
throw new IOException("error happens when parsing create routine load stmt: " + origStmt, e);
}
}
+
+ abstract public void modifyProperties(AlterRoutineLoadStmt stmt) throws DdlException;
Review comment:
The job properties maybe could be modified in here instead of the subclass `KafkaRoutineLoad`.
##########
File path: fe/fe-core/src/main/cup/sql_parser.cup
##########
@@ -722,11 +724,26 @@ alter_stmt ::=
{:
RESULT = new AlterDatabaseQuotaStmt(dbName, QuotaType.REPLICA, String.valueOf(number));
:}
-
| KW_ALTER KW_DATABASE ident:dbName KW_RENAME ident:newDbName
{:
RESULT = new AlterDatabaseRename(dbName, newDbName);
:}
+ | KW_ALTER KW_ROUTINE KW_LOAD job_label:jobLabel opt_properties:jobProperties
Review comment:
Maybe `KW_ALTER KW_ROUTINE KW_LOAD KW_FOR job_label:jobLabel ` is better. The other routine load stmt also has KW_FOR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] wutiangan commented on a change in pull request #4158: [RoutineLoad] Support modify routine load job
Posted by GitBox <gi...@apache.org>.
wutiangan commented on a change in pull request #4158:
URL: https://github.com/apache/incubator-doris/pull/4158#discussion_r459284714
##########
File path: fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadManager.java
##########
@@ -602,6 +573,24 @@ public void replayChangeRoutineLoadJob(RoutineLoadOperation operation) {
.add("msg", "replay change routine load job")
.build());
}
+
+ /**
+ * Enter of altering a routine load job
+ */
+ public void alterRoutineLoadJob(AlterRoutineLoadStmt stmt) throws UserException {
+ RoutineLoadJob job = checkPrivAndGetJob(stmt.getDbName(), stmt.getLabel());
+ if (stmt.hasDataSourceProperty()
+ && !stmt.getDataSourceProperties().getType().equalsIgnoreCase(job.dataSourceType.name())) {
+ throw new DdlException("The spciefied job type is not: " + stmt.getDataSourceProperties().getType());
Review comment:
‘spciefied’ Wrong characters
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] wutiangan commented on a change in pull request #4158: [RoutineLoad] Support modify routine load job
Posted by GitBox <gi...@apache.org>.
wutiangan commented on a change in pull request #4158:
URL: https://github.com/apache/incubator-doris/pull/4158#discussion_r459291033
##########
File path: fe/fe-core/src/main/cup/sql_parser.cup
##########
@@ -722,11 +724,26 @@ alter_stmt ::=
{:
RESULT = new AlterDatabaseQuotaStmt(dbName, QuotaType.REPLICA, String.valueOf(number));
:}
-
| KW_ALTER KW_DATABASE ident:dbName KW_RENAME ident:newDbName
{:
RESULT = new AlterDatabaseRename(dbName, newDbName);
:}
+ | KW_ALTER KW_ROUTINE KW_LOAD job_label:jobLabel opt_properties:properties
Review comment:
```suggestion
| KW_ALTER KW_ROUTINE KW_LOAD job_label:jobLabel opt_properties:jobProperties
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #4158: [RoutineLoad] Support modify routine load job
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #4158:
URL: https://github.com/apache/incubator-doris/pull/4158#discussion_r459432795
##########
File path: fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadManager.java
##########
@@ -602,6 +573,24 @@ public void replayChangeRoutineLoadJob(RoutineLoadOperation operation) {
.add("msg", "replay change routine load job")
.build());
}
+
+ /**
+ * Enter of altering a routine load job
+ */
+ public void alterRoutineLoadJob(AlterRoutineLoadStmt stmt) throws UserException {
+ RoutineLoadJob job = checkPrivAndGetJob(stmt.getDbName(), stmt.getLabel());
+ if (stmt.hasDataSourceProperty()
+ && !stmt.getDataSourceProperties().getType().equalsIgnoreCase(job.dataSourceType.name())) {
+ throw new DdlException("The spciefied job type is not: " + stmt.getDataSourceProperties().getType());
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #4158: [RoutineLoad] Support modify routine load job
Posted by GitBox <gi...@apache.org>.
EmmyMiao87 commented on a change in pull request #4158:
URL: https://github.com/apache/incubator-doris/pull/4158#discussion_r464295130
##########
File path: docs/.vuepress/sidebar/zh-CN.js
##########
@@ -409,6 +409,7 @@ module.exports = [
title: "DML",
directoryPath: "Data Manipulation/",
children: [
+ "alter-routine-load",
Review comment:
Put it between pause routine load and stop routine load.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #4158: [RoutineLoad] Support modify routine load job
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #4158:
URL: https://github.com/apache/incubator-doris/pull/4158#discussion_r459432282
##########
File path: fe/fe-core/src/main/cup/sql_parser.cup
##########
@@ -722,11 +724,26 @@ alter_stmt ::=
{:
RESULT = new AlterDatabaseQuotaStmt(dbName, QuotaType.REPLICA, String.valueOf(number));
:}
-
| KW_ALTER KW_DATABASE ident:dbName KW_RENAME ident:newDbName
{:
RESULT = new AlterDatabaseRename(dbName, newDbName);
:}
+ | KW_ALTER KW_ROUTINE KW_LOAD job_label:jobLabel opt_properties:properties
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #4158: [RoutineLoad] Support modify routine load job
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #4158:
URL: https://github.com/apache/incubator-doris/pull/4158#discussion_r464313843
##########
File path: fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java
##########
@@ -1397,4 +1401,36 @@ public void readFields(DataInput in) throws IOException {
throw new IOException("error happens when parsing create routine load stmt: " + origStmt, e);
}
}
+
+ abstract public void modifyProperties(AlterRoutineLoadStmt stmt) throws DdlException;
Review comment:
Maybe we could do this when we got second derived class from RoutineLoadJob
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #4158: [RoutineLoad] Support modify routine load job
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #4158:
URL: https://github.com/apache/incubator-doris/pull/4158#discussion_r464313344
##########
File path: fe/fe-core/src/main/cup/sql_parser.cup
##########
@@ -722,11 +724,26 @@ alter_stmt ::=
{:
RESULT = new AlterDatabaseQuotaStmt(dbName, QuotaType.REPLICA, String.valueOf(number));
:}
-
| KW_ALTER KW_DATABASE ident:dbName KW_RENAME ident:newDbName
{:
RESULT = new AlterDatabaseRename(dbName, newDbName);
:}
+ | KW_ALTER KW_ROUTINE KW_LOAD job_label:jobLabel opt_properties:jobProperties
Review comment:
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #4158: [RoutineLoad] Support modify routine load job
Posted by GitBox <gi...@apache.org>.
morningman merged pull request #4158:
URL: https://github.com/apache/incubator-doris/pull/4158
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org