You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/04/28 11:08:36 UTC
[GitHub] [dolphinscheduler] Dannila opened a new pull request, #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Dannila opened a new pull request, #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840
<!--Thanks very much for contributing to Apache DolphinScheduler. Please review https://dolphinscheduler.apache.org/en-us/community/development/pull-request.html before opening a pull request.-->
## Purpose of the pull request
We want FlinkTask to have an editor to fill out flink sql.
Users can implement flink sql tasks or flink sql cdc tasks through the sql editor, which is convenient for users to perform data processing and ETL.
This change added tests and can be verified as follows:
<!--(For example: This pull request adds checkstyle plugin).-->
## Brief change log
ProgramType.class add sql type
FlinkTask.class and FlinkTask.class add main logical for support Spark sql command
FlinkConstants.class 和 FlinkParameters.class Common parameters for connecting Flink sql commands and methods for obtaining parameters
use-flink.ts adds some logic to support front-end pages supporting spark sql editing SQL
<!--*(for example:)*
- *Add maven-checkstyle-plugin to root pom.xml*
-->
## Verify this pull request
This change added tests and can be verified as follows:
org.apache.dolphinscheduler.plugin.task.flink.FlinkTaskTest
<!--*(Please pick either of the following options)*-->
This pull request is code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
<!--*(example:)*
- *Added dolphinscheduler-dao tests for end-to-end.*
- *Added CronUtilsTest to verify the change.*
- *Manually verified the change by testing locally.* -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1119413037
> Please `rebase` the latest code, we removed the old `UI`.
Yes, in the latest commit, I have `rebase` the latest code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1120946410
Thanks for your contribution. This is a good beginning. Considering that it's your first contribution, I think we can get deep communication, you can contact me through mail or add WeChat(SbloodyS), when mail or added, please tell me who you are. I think I can help to familiar with the Dolphinscheduler if you meet with problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] devosend commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
devosend commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1120581887
To ensure the quality and style of the front-end code, you should run ` pnpm run prettier` to format the front-end code. @Dannila
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] devosend commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
devosend commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861427086
##########
dolphinscheduler-ui-next/src/views/projects/task/components/node/fields/use-flink.ts:
##########
@@ -14,15 +14,27 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-import { computed, ref } from 'vue'
+import { computed, ref , watchEffect} from 'vue'
Review Comment:
Prettier format code should be used here.
##########
dolphinscheduler-ui-next/src/views/projects/task/components/node/fields/use-flink.ts:
##########
@@ -14,15 +14,27 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-import { computed, ref } from 'vue'
+import { computed, ref , watchEffect} from 'vue'
import { useI18n } from 'vue-i18n'
import { useCustomParams, useDeployMode, useMainJar, useResources } from '.'
import type { IJsonItem } from '../types'
export function useFlink(model: { [field: string]: any }): IJsonItem[] {
const { t } = useI18n()
const mainClassSpan = computed(() =>
- model.programType === 'PYTHON' ? 0 : 24
+ (model.programType === 'PYTHON' || model.programType === 'SQL') ? 0 : 24
+ )
+
+ const mainArgsSpan = computed(() =>
+ model.programType === 'SQL' ? 0 : 24
+ )
+
+ const rawScriptSpan = computed(() =>
+ model.programType === 'SQL' ? 24 : 0
+ )
+
+ const flinkVersionOptions = computed(() =>
Review Comment:
above
##########
dolphinscheduler-ui-next/src/views/projects/task/components/node/fields/use-flink.ts:
##########
@@ -59,21 +75,32 @@ export function useFlink(model: { [field: string]: any }): IJsonItem[] {
},
validate: {
trigger: ['input', 'blur'],
- required: model.programType !== 'PYTHON',
+ required: model.programType !== 'PYTHON' && model.programType !== 'SQL',
validator(validate: any, value: string) {
- if (model.programType !== 'PYTHON' && !value) {
+ if (model.programType !== 'PYTHON' && !value && model.programType !== 'SQL') {
return new Error(t('project.node.main_class_tips'))
}
}
}
},
useMainJar(model),
Review Comment:
You should judge programType in `useMainJar `. It should not request `queryResourceByProgramType` when programType is SQL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1112868751
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9840)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [3 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list) [72.1% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list) [3.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] devosend commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
devosend commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861559748
##########
dolphinscheduler-ui-next/src/views/projects/task/components/node/fields/use-flink.ts:
##########
@@ -14,15 +14,27 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-import { computed, ref } from 'vue'
+import { computed, ref , watchEffect} from 'vue'
Review Comment:
You can install prettier plugin for editor or run `pnpm run prettier`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1117335801
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9840)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [2 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list) [72.2% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list) [2.5% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861435242
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkConstants.java:
##########
@@ -24,19 +24,48 @@ private FlinkConstants() {
}
/**
- * flink
+ * flink command
+ * usage: flink run [OPTIONS] <jar-file> <arguments>
+ */
+ public static final String FLINK_COMMAND = "flink";
+ public static final String FLINK_RUN = "run";
+
+ /**
+ * flink sql command
+ * usage: sql-client.sh -i <initialization file>, -j <JAR file>, -f <script file>...
+ */
+ public static final String FLINK_SQL_COMMAND = "sql-client.sh";
Review Comment:
Users want us to support flink sql as soon as possible. When the user enters the sql script in the editor, we roll it back to a file and execute this file with flink `sql-client.sh -f`. Features like `-i` or `--jar` are not included for now. Supplements can be added later if desired
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] labbomb commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
labbomb commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1112802282
@Amy0104 @devosend Take a look at this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861485941
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkConstants.java:
##########
@@ -24,19 +24,48 @@ private FlinkConstants() {
}
/**
- * flink
+ * flink command
+ * usage: flink run [OPTIONS] <jar-file> <arguments>
+ */
+ public static final String FLINK_COMMAND = "flink";
+ public static final String FLINK_RUN = "run";
+
+ /**
+ * flink sql command
+ * usage: sql-client.sh -i <initialization file>, -j <JAR file>, -f <script file>...
+ */
+ public static final String FLINK_SQL_COMMAND = "sql-client.sh";
Review Comment:
For the first solution above, the current pr already satisfies the function of `-i`, do you think you need to continue to modify it?
Of course I'm more than willing to optimize the second solution, and then I can continue to optimize in the next pr.
It's worth noting that documentation is being added, please wait.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861417848
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkConstants.java:
##########
@@ -24,19 +24,48 @@ private FlinkConstants() {
}
/**
- * flink
+ * flink command
+ * usage: flink run [OPTIONS] <jar-file> <arguments>
+ */
+ public static final String FLINK_COMMAND = "flink";
+ public static final String FLINK_RUN = "run";
+
+ /**
+ * flink sql command
+ * usage: sql-client.sh -i <initialization file>, -j <JAR file>, -f <script file>...
+ */
+ public static final String FLINK_SQL_COMMAND = "sql-client.sh";
Review Comment:
Please note that flink ```sql-client.sh -f``` is only supported after version 1.13. And ```sql-client.sh``` at version 1.13 and 1.14. and 1.15 have some completely incompatible contents in these three versions. If we add flink sql support. We should put it in the frontend page and document to let users distinguish it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] songjianet commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
songjianet commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1119414744
> > Please `rebase` the latest code, we removed the old `UI`.
>
> Yes, in the latest commit, I have `rebase` the latest code.
It may be an operation problem. We removed the old `UI` and renamed `UI-Next` to `UI`, so from the current point of view, the operation on your side has not been successful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861440145
##########
dolphinscheduler-ui-next/src/views/projects/task/components/node/fields/use-flink.ts:
##########
@@ -59,21 +75,32 @@ export function useFlink(model: { [field: string]: any }): IJsonItem[] {
},
validate: {
trigger: ['input', 'blur'],
- required: model.programType !== 'PYTHON',
+ required: model.programType !== 'PYTHON' && model.programType !== 'SQL',
validator(validate: any, value: string) {
- if (model.programType !== 'PYTHON' && !value) {
+ if (model.programType !== 'PYTHON' && !value && model.programType !== 'SQL') {
return new Error(t('project.node.main_class_tips'))
}
}
}
},
useMainJar(model),
Review Comment:
When the user clicks the programType as sql on the front end, the mainClass will be removed, and the edit box will be displayed for the user to fill in the sql
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861494785
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkConstants.java:
##########
@@ -24,19 +24,48 @@ private FlinkConstants() {
}
/**
- * flink
+ * flink command
+ * usage: flink run [OPTIONS] <jar-file> <arguments>
+ */
+ public static final String FLINK_COMMAND = "flink";
+ public static final String FLINK_RUN = "run";
+
+ /**
+ * flink sql command
+ * usage: sql-client.sh -i <initialization file>, -j <JAR file>, -f <script file>...
+ */
+ public static final String FLINK_SQL_COMMAND = "sql-client.sh";
Review Comment:
The front end provides two editors, one for filling in the user's initialization script and the other for filling in FlinkSQL.
then I will continue to optimize this pr according to this plan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] AirToSupply commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
AirToSupply commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861629706
##########
dolphinscheduler-ui-next/src/views/projects/task/components/node/fields/use-flink.ts:
##########
@@ -59,21 +75,32 @@ export function useFlink(model: { [field: string]: any }): IJsonItem[] {
},
validate: {
trigger: ['input', 'blur'],
- required: model.programType !== 'PYTHON',
+ required: model.programType !== 'PYTHON' && model.programType !== 'SQL',
validator(validate: any, value: string) {
- if (model.programType !== 'PYTHON' && !value) {
+ if (model.programType !== 'PYTHON' && !value && model.programType !== 'SQL') {
return new Error(t('project.node.main_class_tips'))
}
}
}
},
useMainJar(model),
Review Comment:
@Dannila modify **use-main-jar.ts** watch function
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1112803057
Hi @Dannila , can you please add this to the docs? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r867291088
##########
docs/docs/zh/guide/task/flink.md:
##########
@@ -2,7 +2,11 @@
## 综述
-Flink 任务类型，用于执行 Flink 程序。对于 Flink 节点，worker 会通过使用 flink 命令 `flink run` 的方式提交任务。更多详情查看 [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/)。
+Flink 任务类型，用于执行 Flink 程序。对于 Flink 节点：
+
+(1) 当程序类型为 Java、Scala 或 Python 时，worker 使用 Flink 命令提交任务 `flink run`。更多详情查看 [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) 。
+
+(2) 当程序类型为 SQL 时，worker 使用 Flink SQL 提交任务 `sql-client.sh`。更多详情查看 [flink sql client](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/) 。
Review Comment:
You're right, I'll fix it now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] AirToSupply commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
AirToSupply commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861476372
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkConstants.java:
##########
@@ -24,19 +24,48 @@ private FlinkConstants() {
}
/**
- * flink
+ * flink command
+ * usage: flink run [OPTIONS] <jar-file> <arguments>
+ */
+ public static final String FLINK_COMMAND = "flink";
+ public static final String FLINK_RUN = "run";
+
+ /**
+ * flink sql command
+ * usage: sql-client.sh -i <initialization file>, -j <JAR file>, -f <script file>...
+ */
+ public static final String FLINK_SQL_COMMAND = "sql-client.sh";
Review Comment:
I get it! I think there are two ways：
（1）Edit the initialization script in [**resource center**], select this resource when creating a task, and then specify -i directly in [**pption parameters**]. This way I just took off this pr to verify that it works.
（2）The front end provides **two** editors, one for filling in the user's initialization script and one for filling in FlinkSQL, which may be a little clearer.
Could you help me with a better solution? We'll just optimize part of code immediately, Grazie！
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] AirToSupply commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
AirToSupply commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861446491
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkConstants.java:
##########
@@ -24,19 +24,48 @@ private FlinkConstants() {
}
/**
- * flink
+ * flink command
+ * usage: flink run [OPTIONS] <jar-file> <arguments>
+ */
+ public static final String FLINK_COMMAND = "flink";
+ public static final String FLINK_RUN = "run";
+
+ /**
+ * flink sql command
+ * usage: sql-client.sh -i <initialization file>, -j <JAR file>, -f <script file>...
+ */
+ public static final String FLINK_SQL_COMMAND = "sql-client.sh";
Review Comment:
【-i】: Script file that used to init the session context. If get error in execution, the sql client will exit.Notice it's not allowed to add query or insert into the init file.
The initialization script specified by -i is used by most users to specify some parameters like set a=b. Of course, these parameters can also be placed in the SQL script specified by -f. So our users also want to set the parameters of flink themselves in the front-end editor. Of course, we also do built-in processing for the flink parameters filled in by the user front-end in the FlinkTask class. These parameters are as follows:
```sql
set sql-client.execution.result-mode=tableau;
set execution.target=yarn-per-job;
set yarn.application.name=streaming-top-speed-windowing-sql;
set yarn.application.queue=default;
set jobmanager.memory.process.size=1024mb;
set taskmanager.memory.process.size=4096mb;
set taskmanager.numberOfTaskSlots=2;
set parallelism.default=4;
set execution.runtime-mode=streaming;
```
At the same time -i can be specified directly under the control of the front-end option parameter ～
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r867296629
##########
docs/docs/zh/guide/task/flink.md:
##########
@@ -64,6 +70,14 @@ Flink 任务类型，用于执行 Flink 程序。对于 Flink 节点，worker
+### 执行 FlinkSQL 程序
+
+根据上述参数说明，配置所需的内容即可。
+
+
+
## 注意事项：
-Java 和 Scala 只是用来标识，没有区别，如果是 Python 开发的 Flink 则没有主函数的 class，其余的都一样。
+- Java 和 Scala 只是用来标识，没有区别，如果是 Python 开发的 Flink 则没有主函数的 class，其余的都一样。
+
+- 使用sql 执行Flink SQL 任务，目前只支持flink1.13及以上版本。
Review Comment:
We should keep the same format with or without spaces. ```使用 SQL 执行 Flink SQL 任务，目前只支持Flink 1.13及以上版本。```
##########
docs/docs/zh/guide/task/flink.md:
##########
@@ -64,6 +70,14 @@ Flink 任务类型，用于执行 Flink 程序。对于 Flink 节点，worker
+### 执行 FlinkSQL 程序
+
+根据上述参数说明，配置所需的内容即可。
+
+
+
## 注意事项：
-Java 和 Scala 只是用来标识，没有区别，如果是 Python 开发的 Flink 则没有主函数的 class，其余的都一样。
+- Java 和 Scala 只是用来标识，没有区别，如果是 Python 开发的 Flink 则没有主函数的 class，其余的都一样。
+
+- 使用sql 执行Flink SQL 任务，目前只支持flink1.13及以上版本。
Review Comment:
We should keep the same format with or without spaces. ```使用 SQL 执行 Flink SQL 任务，目前只支持 Flink 1.13 及以上版本。```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1120597937
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9840)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [2 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list) [74.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list) [2.2% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1120584212
>
could we add it to CI? when could not formatted will fail our CI @devosend ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1118120590
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9840)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [2 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list) [71.8% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list) [2.2% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1112808561
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/9840?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#9840](https://codecov.io/gh/apache/dolphinscheduler/pull/9840?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7850a08) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/828034f8b9760ed51cd5657e31b4e4f462c643c0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (828034f) will **increase** coverage by `0.34%`.
> The diff coverage is `62.04%`.
> :exclamation: Current head 7850a08 differs from pull request most recent head fa5e9d0. Consider uploading reports for the commit fa5e9d0 to get more accurate results
```diff
@@ Coverage Diff @@
## dev #9840 +/- ##
============================================
+ Coverage 40.41% 40.75% +0.34%
- Complexity 4527 4569 +42
============================================
Files 834 833 -1
Lines 33639 33700 +61
Branches 3721 3737 +16
============================================
+ Hits 13595 13736 +141
+ Misses 18760 18648 -112
- Partials 1284 1316 +32
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/9840?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...er/server/master/runner/WorkflowExecuteThread.java](https://codecov.io/gh/apache/dolphinscheduler/pull/9840/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1tYXN0ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9ydW5uZXIvV29ya2Zsb3dFeGVjdXRlVGhyZWFkLmphdmE=) | `8.29% <0.00%> (ø)` | |
| [...hinscheduler/plugin/task/flink/FlinkConstants.java](https://codecov.io/gh/apache/dolphinscheduler/pull/9840/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci10YXNrLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLXRhc2stZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcGx1Z2luL3Rhc2svZmxpbmsvRmxpbmtDb25zdGFudHMuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...hinscheduler/plugin/alert/script/ScriptSender.java](https://codecov.io/gh/apache/dolphinscheduler/pull/9840/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1hbGVydC9kb2xwaGluc2NoZWR1bGVyLWFsZXJ0LXBsdWdpbnMvZG9scGhpbnNjaGVkdWxlci1hbGVydC1zY3JpcHQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcGx1Z2luL2FsZXJ0L3NjcmlwdC9TY3JpcHRTZW5kZXIuamF2YQ==) | `60.60% <11.11%> (-18.57%)` | :arrow_down: |
| [.../dolphinscheduler/plugin/task/flink/FlinkTask.java](https://codecov.io/gh/apache/dolphinscheduler/pull/9840/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci10YXNrLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLXRhc2stZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcGx1Z2luL3Rhc2svZmxpbmsvRmxpbmtUYXNrLmphdmE=) | `65.24% <65.28%> (+65.24%)` | :arrow_up: |
| [...inscheduler/plugin/task/flink/FlinkParameters.java](https://codecov.io/gh/apache/dolphinscheduler/pull/9840/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci10YXNrLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLXRhc2stZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcGx1Z2luL3Rhc2svZmxpbmsvRmxpbmtQYXJhbWV0ZXJzLmphdmE=) | `98.14% <75.00%> (+80.50%)` | :arrow_up: |
| [...olphinscheduler/plugin/task/flink/ProgramType.java](https://codecov.io/gh/apache/dolphinscheduler/pull/9840/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci10YXNrLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLXRhc2stZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcGx1Z2luL3Rhc2svZmxpbmsvUHJvZ3JhbVR5cGUuamF2YQ==) | `100.00% <100.00%> (+100.00%)` | :arrow_up: |
| [...er/master/dispatch/host/assign/RandomSelector.java](https://codecov.io/gh/apache/dolphinscheduler/pull/9840/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1tYXN0ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9kaXNwYXRjaC9ob3N0L2Fzc2lnbi9SYW5kb21TZWxlY3Rvci5qYXZh) | `77.77% <0.00%> (-5.56%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/9840?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/9840?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [828034f...fa5e9d0](https://codecov.io/gh/apache/dolphinscheduler/pull/9840?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861441052
##########
dolphinscheduler-ui-next/src/views/projects/task/components/node/fields/use-flink.ts:
##########
@@ -14,15 +14,27 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-import { computed, ref } from 'vue'
+import { computed, ref , watchEffect} from 'vue'
Review Comment:
WatchEffect is added here to monitor the value of programType and let flinkVersion change dynamically
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1118096680
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9840)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [2 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list) [71.8% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list) [2.2% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1120120683
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9840)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [2 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list) [74.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list) [2.2% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r867296629
##########
docs/docs/zh/guide/task/flink.md:
##########
@@ -64,6 +70,14 @@ Flink 任务类型，用于执行 Flink 程序。对于 Flink 节点，worker
+### 执行 FlinkSQL 程序
+
+根据上述参数说明，配置所需的内容即可。
+
+
+
## 注意事项：
-Java 和 Scala 只是用来标识，没有区别，如果是 Python 开发的 Flink 则没有主函数的 class，其余的都一样。
+- Java 和 Scala 只是用来标识，没有区别，如果是 Python 开发的 Flink 则没有主函数的 class，其余的都一样。
+
+- 使用sql 执行Flink SQL 任务，目前只支持flink1.13及以上版本。
Review Comment:
We should keep the same format with or without spaces. ```使用 SQL 执行 Flink SQL 任务```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r867601341
##########
docs/docs/en/guide/task/flink.md:
##########
@@ -2,7 +2,11 @@
## Overview
-Flink task type for executing Flink programs. For Flink nodes, the worker submits the task by using the Flink command `flink run`. See [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) for more details.
+Flink task type, used to execute Flink programs. For Flink nodes:
+
+(1) When the program type is Java, Scala or Python, the worker submits the task `flink run` using the Flink command. See [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) for more details.
+
+(2) When the program type is SQL, the worker submit tasks using `sql-client.sh`. See [flink sql client](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/) for more details.
Review Comment:
ok， thank
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r867600010
##########
docs/docs/en/guide/task/flink.md:
##########
@@ -2,7 +2,11 @@
## Overview
-Flink task type for executing Flink programs. For Flink nodes, the worker submits the task by using the Flink command `flink run`. See [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) for more details.
+Flink task type, used to execute Flink programs. For Flink nodes:
+
+(1) When the program type is Java, Scala or Python, the worker submits the task `flink run` using the Flink command. See [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) for more details.
+
+(2) When the program type is SQL, the worker submit tasks using `sql-client.sh`. See [flink sql client](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/) for more details.
Review Comment:
I will modify it now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1120108217
> > > 请`rebase`最新的代码，我们删除旧的`UI`。
> >
> >
> > 是的，在最新的提交中，我有`rebase`最新的代码。
>
> 可能是操作。我们取消了旧的`UI`改写，所以从目前`UI-Next`来看`UI`，你的这个操作没有成功。
Excuse me, is this operation successful in the latest commit?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r867294854
##########
docs/docs/en/guide/task/flink.md:
##########
@@ -2,7 +2,11 @@
## Overview
-Flink task type for executing Flink programs. For Flink nodes, the worker submits the task by using the Flink command `flink run`. See [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) for more details.
+Flink task type, used to execute Flink programs. For Flink nodes:
+
+(1) When the program type is Java, Scala or Python, the worker submits the task `flink run` using the Flink command. See [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) for more details.
+
+(2) When the program type is Sql, the worker submits the task `sql-client.sh` using Flink Sql. See [flink sql client](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/) for more details.
Review Comment:
This has not been addressed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] devosend commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
devosend commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861557163
##########
dolphinscheduler-ui-next/src/views/projects/task/components/node/fields/use-flink.ts:
##########
@@ -59,21 +75,32 @@ export function useFlink(model: { [field: string]: any }): IJsonItem[] {
},
validate: {
trigger: ['input', 'blur'],
- required: model.programType !== 'PYTHON',
+ required: model.programType !== 'PYTHON' && model.programType !== 'SQL',
validator(validate: any, value: string) {
- if (model.programType !== 'PYTHON' && !value) {
+ if (model.programType !== 'PYTHON' && !value && model.programType !== 'SQL') {
return new Error(t('project.node.main_class_tips'))
}
}
}
},
useMainJar(model),
Review Comment:
> When the user clicks the programType as sql on the front end, the mainClass will be removed, and the edit box will be displayed for the user to fill in the sql
But it still call `queryResourceByProgramType` func and send a request to the backend.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1112846535
> ok, another pr will be mentioned later to add documents
Please also add documents in this PR. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] AirToSupply commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
AirToSupply commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1112856071
>
I think it needs to be clearer！
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r867598431
##########
docs/docs/en/guide/task/flink.md:
##########
@@ -2,7 +2,11 @@
## Overview
-Flink task type for executing Flink programs. For Flink nodes, the worker submits the task by using the Flink command `flink run`. See [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) for more details.
+Flink task type, used to execute Flink programs. For Flink nodes:
+
+(1) When the program type is Java, Scala or Python, the worker submits the task `flink run` using the Flink command. See [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) for more details.
+
+(2) When the program type is SQL, the worker submit tasks using `sql-client.sh`. See [flink sql client](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/) for more details.
Review Comment:
```suggestion
1. When the program type is Java, Scala or Python, the worker submits the task `flink run` using the Flink command. See [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) for more details.
2. When the program type is SQL, the worker submit tasks using `sql-client.sh`. See [flink sql client](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/) for more details.
```
##########
docs/docs/zh/guide/task/flink.md:
##########
@@ -2,7 +2,11 @@
## 综述
-Flink 任务类型，用于执行 Flink 程序。对于 Flink 节点，worker 会通过使用 flink 命令 `flink run` 的方式提交任务。更多详情查看 [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/)。
+Flink 任务类型，用于执行 Flink 程序。对于 Flink 节点：
+
+(1) 当程序类型为 Java、Scala 或 Python 时，worker 使用 Flink 命令提交任务 `flink run`。更多详情查看 [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) 。
+
+(2) 当程序类型为 SQL 时，worker 使用`sql-client.sh` 提交任务。更多详情查看 [flink sql client](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/) 。
Review Comment:
```suggestion
1. 当程序类型为 Java、Scala 或 Python 时，worker 使用 Flink 命令提交任务 `flink run`。更多详情查看 [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) 。
2. 当程序类型为 SQL 时，worker 使用`sql-client.sh` 提交任务。更多详情查看 [flink sql client](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/) 。
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] dailidong commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
dailidong commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1117323089
> > ok, another pr will be mentioned later to add documents
>
> Please also add documents in this PR. Thanks.
good job, supporting Flink sql is big feature, By the way, the doc is very important, the PR with relevant doc will keep the code and documentation as consistent as possible
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1119359974
> @Dannila Please resolve conflicts.
OK, I'll fix it in the next commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1119629575
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9840)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [2 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list) [74.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list) [2.2% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1118112356
> Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9840)
>
> [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [2 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL)
>
> [](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list) [71.8% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list) [](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&vi
ew=list) [2.2% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list)
BTW, please check the https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1118111643
Well done, I had approved to run CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1120108587
> >
>
> yes, will review later.
Gratitude！
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1119340400
@Dannila Please resolve conflicts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1120586199
we should have frontend get at least one approval from our frontend developer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1112810899
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9840)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list) [72.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list) [3.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861423199
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkConstants.java:
##########
@@ -24,19 +24,48 @@ private FlinkConstants() {
}
/**
- * flink
+ * flink command
+ * usage: flink run [OPTIONS] <jar-file> <arguments>
+ */
+ public static final String FLINK_COMMAND = "flink";
+ public static final String FLINK_RUN = "run";
+
+ /**
+ * flink sql command
+ * usage: sql-client.sh -i <initialization file>, -j <JAR file>, -f <script file>...
+ */
+ public static final String FLINK_SQL_COMMAND = "sql-client.sh";
Review Comment:
Flink sql requires sql-client.sh -f <script file> to facilitate users to edit sql scripts. I set it on the front end. When the user selects the flink task and selects sql, the version of flink is >= 1.13 and cannot be changed. flink1.15 also supports sql-client.sh -f
【flink 1.15 sql client】https://nightlies.apache.org/flink/flink-docs-release-1.15/zh/docs/dev/table/sqlclient/
【flink 1.16 sql client】https://nightlies.apache.org/flink/flink-docs-release-1.16/zh/docs/dev/table/sqlclient/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861489999
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkConstants.java:
##########
@@ -24,19 +24,48 @@ private FlinkConstants() {
}
/**
- * flink
+ * flink command
+ * usage: flink run [OPTIONS] <jar-file> <arguments>
+ */
+ public static final String FLINK_COMMAND = "flink";
+ public static final String FLINK_RUN = "run";
+
+ /**
+ * flink sql command
+ * usage: sql-client.sh -i <initialization file>, -j <JAR file>, -f <script file>...
+ */
+ public static final String FLINK_SQL_COMMAND = "sql-client.sh";
Review Comment:
> For the first solution above, the current pr already satisfies the function of `-i`, do you think you need to continue to modify it?
>
> Of course I'm more than willing to optimize the second solution, and then I can continue to optimize in the next pr.
>
> It's worth noting that documentation is being added, please wait.
It would be friendly if you could enlight me where you implement ```-i``` to enable users to specify it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861428918
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkConstants.java:
##########
@@ -24,19 +24,48 @@ private FlinkConstants() {
}
/**
- * flink
+ * flink command
+ * usage: flink run [OPTIONS] <jar-file> <arguments>
+ */
+ public static final String FLINK_COMMAND = "flink";
+ public static final String FLINK_RUN = "run";
+
+ /**
+ * flink sql command
+ * usage: sql-client.sh -i <initialization file>, -j <JAR file>, -f <script file>...
+ */
+ public static final String FLINK_SQL_COMMAND = "sql-client.sh";
Review Comment:
Yes. I think we should also add the new features of Flink 1.13 to flink task. For example ```-i``` from UI or resource center and ```--jar``` from resource center. In particular, I think the ```-i``` option will be used in most scenarios when users use multiple connectors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861441142
##########
dolphinscheduler-ui-next/src/views/projects/task/components/node/fields/use-flink.ts:
##########
@@ -14,15 +14,27 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-import { computed, ref } from 'vue'
+import { computed, ref , watchEffect} from 'vue'
Review Comment:
I don't have a better way to modify it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] songjianet commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
songjianet commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1119408858
Please `rebase` the latest code, we removed the old `UI`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r867297472
##########
docs/docs/zh/guide/task/flink.md:
##########
@@ -64,6 +70,14 @@ Flink 任务类型，用于执行 Flink 程序。对于 Flink 节点，worker
+### 执行 FlinkSQL 程序
+
+根据上述参数说明，配置所需的内容即可。
+
+
+
## 注意事项：
-Java 和 Scala 只是用来标识，没有区别，如果是 Python 开发的 Flink 则没有主函数的 class，其余的都一样。
+- Java 和 Scala 只是用来标识，没有区别，如果是 Python 开发的 Flink 则没有主函数的 class，其余的都一样。
+
+- 使用sql 执行Flink SQL 任务，目前只支持flink1.13及以上版本。
Review Comment:
Thanks a lot, I'll check again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1120131354
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9840)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [2 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list) [74.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list) [2.2% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r867600477
##########
docs/docs/en/guide/task/flink.md:
##########
@@ -2,7 +2,11 @@
## Overview
-Flink task type for executing Flink programs. For Flink nodes, the worker submits the task by using the Flink command `flink run`. See [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) for more details.
+Flink task type, used to execute Flink programs. For Flink nodes:
+
+(1) When the program type is Java, Scala or Python, the worker submits the task `flink run` using the Flink command. See [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) for more details.
+
+(2) When the program type is SQL, the worker submit tasks using `sql-client.sh`. See [flink sql client](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/) for more details.
Review Comment:
> I will modify it now
This has been modified by @zhongjiajie . There is no need to change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1120587079
> To ensure the quality and style of the front-end code, you should run ` pnpm run prettier` to format the front-end code. @Dannila
Sorry, I don't know much about front-end, I don't know much about this `should run pnpm run prettier to format front-end code`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1120943131
>The front end part LGTM
Greatful
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1120945860
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9840)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [2 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list) [74.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list) [2.2% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS merged pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS merged PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861422745
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkConstants.java:
##########
@@ -24,19 +24,48 @@ private FlinkConstants() {
}
/**
- * flink
+ * flink command
+ * usage: flink run [OPTIONS] <jar-file> <arguments>
+ */
+ public static final String FLINK_COMMAND = "flink";
+ public static final String FLINK_RUN = "run";
+
+ /**
+ * flink sql command
+ * usage: sql-client.sh -i <initialization file>, -j <JAR file>, -f <script file>...
+ */
+ public static final String FLINK_SQL_COMMAND = "sql-client.sh";
Review Comment:
> Please note that flink `sql-client.sh -f` is only supported after version 1.13. And `sql-client.sh` at version 1.13 and 1.14. and 1.15 have some completely incompatible contents in these three versions. If we add flink sql support. We should put it in the frontend page and document to let users distinguish it.
Flink sql requires sql-client.sh -f <script file> to facilitate users to edit sql scripts. I set it on the front end. When the user selects the flink task and selects sql, the version of flink is >= 1.13 and cannot be changed. flink1.15 also supports sql-client.sh -f https://nightlies.apache.org/flink/flink-docs-release-1.15/zh/docs/dev/table/sqlclient/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1112813664
> 你好@Dannila，你能把这个添加到文档中吗？谢谢。
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861448043
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkConstants.java:
##########
@@ -24,19 +24,48 @@ private FlinkConstants() {
}
/**
- * flink
+ * flink command
+ * usage: flink run [OPTIONS] <jar-file> <arguments>
+ */
+ public static final String FLINK_COMMAND = "flink";
+ public static final String FLINK_RUN = "run";
+
+ /**
+ * flink sql command
+ * usage: sql-client.sh -i <initialization file>, -j <JAR file>, -f <script file>...
+ */
+ public static final String FLINK_SQL_COMMAND = "sql-client.sh";
Review Comment:
> 【-i】: Script file that used to init the session context. If get error in execution, the sql client will exit.Notice it's not allowed to add query or insert into the init file.
>
> The initialization script specified by -i is used by most users to specify some parameters like set a=b. Of course, these parameters can also be placed in the SQL script specified by -f. So our users also want to set the parameters of flink themselves in the front-end editor. Of course, we also do built-in processing for the flink parameters filled in by the user front-end in the FlinkTask class. These parameters are as follows:
>
> ```sql
> set sql-client.execution.result-mode=tableau;
> set execution.target=yarn-per-job;
> set yarn.application.name=streaming-top-speed-windowing-sql;
> set yarn.application.queue=default;
> set jobmanager.memory.process.size=1024mb;
> set taskmanager.memory.process.size=4096mb;
> set taskmanager.numberOfTaskSlots=2;
> set parallelism.default=4;
> set execution.runtime-mode=streaming;
> ```
>
> At the same time -i can be specified directly under the control of the front-end option parameter ～
I do not think so. I think the initialization script specified by ```-i``` is used by most users to using different connectors. Users will execute ```DDL``` or ```USE CATALOG``` scripts in order to create a ```flink-cdc``` or ```kafka``` table and use it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] devosend commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
devosend commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861555287
##########
dolphinscheduler-ui-next/src/views/projects/task/components/node/fields/use-flink.ts:
##########
@@ -14,15 +14,27 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-import { computed, ref } from 'vue'
+import { computed, ref , watchEffect} from 'vue'
Review Comment:
> I don't have a better way to modify it
this is a matter of code style. `import { computed, ref, watchEffect} from 'vue'`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861437057
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkConstants.java:
##########
@@ -24,19 +24,48 @@ private FlinkConstants() {
}
/**
- * flink
+ * flink command
+ * usage: flink run [OPTIONS] <jar-file> <arguments>
+ */
+ public static final String FLINK_COMMAND = "flink";
+ public static final String FLINK_RUN = "run";
+
+ /**
+ * flink sql command
+ * usage: sql-client.sh -i <initialization file>, -j <JAR file>, -f <script file>...
+ */
+ public static final String FLINK_SQL_COMMAND = "sql-client.sh";
Review Comment:
```-jar``` supplement can be added later is ok to me. However, almost 90% of the Flink SQL related operations need to be performed based on ```-i```. I think it's needed in flink sql feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r861480817
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkConstants.java:
##########
@@ -24,19 +24,48 @@ private FlinkConstants() {
}
/**
- * flink
+ * flink command
+ * usage: flink run [OPTIONS] <jar-file> <arguments>
+ */
+ public static final String FLINK_COMMAND = "flink";
+ public static final String FLINK_RUN = "run";
+
+ /**
+ * flink sql command
+ * usage: sql-client.sh -i <initialization file>, -j <JAR file>, -f <script file>...
+ */
+ public static final String FLINK_SQL_COMMAND = "sql-client.sh";
Review Comment:
> I get it! I think there are two ways： （1）Edit the initialization script in [**resource center**], select this resource when creating a task, and then specify -i directly in [**pption parameters**]. This way I just took off this pr to verify that it works. （2）The front end provides **two** editors, one for filling in the user's initialization script and one for filling in FlinkSQL, which may be a little clearer. Could you help me with a better solution? We'll just optimize part of code immediately, Grazie！
Since we already have a sql editor for FlinkSQL's ```DML``` scripts. I think the second option is friendly to both users and first-time contributor. We can implement the first option in the next PR and provide it to users if you wish.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1112906510
> 还请在此 PR 中添加文档。谢谢。
Quindi aggiungerò il documento ora
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1119437874
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9840)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [2 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list) [74.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list) [2.2% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1118202700
@zhuangchong Excuse me, do you think this PR needs to be further optimized?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r867287911
##########
docs/docs/zh/guide/task/flink.md:
##########
@@ -2,7 +2,11 @@
## 综述
-Flink 任务类型，用于执行 Flink 程序。对于 Flink 节点，worker 会通过使用 flink 命令 `flink run` 的方式提交任务。更多详情查看 [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/)。
+Flink 任务类型，用于执行 Flink 程序。对于 Flink 节点：
+
+(1) 当程序类型为 Java、Scala 或 Python 时，worker 使用 Flink 命令提交任务 `flink run`。更多详情查看 [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) 。
+
+(2) 当程序类型为 SQL 时，worker 使用 Flink SQL 提交任务 `sql-client.sh`。更多详情查看 [flink sql client](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/) 。
Review Comment:
Is it better to use ```worker 使用 `sql-client.sh` 来提交 Flink SQL 任务```? The same as en's doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] songjianet commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
songjianet commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1120108481
>
yes, will review later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#issuecomment-1120125327
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9840)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT) [2 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9840&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list) [74.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list) [2.2% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9840&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Dannila commented on a diff in pull request #9840: [Feature- 9837][plugin/ui] support FlinkSQL Task
Posted by GitBox <gi...@apache.org>.
Dannila commented on code in PR #9840:
URL: https://github.com/apache/dolphinscheduler/pull/9840#discussion_r867295376
##########
docs/docs/en/guide/task/flink.md:
##########
@@ -2,7 +2,11 @@
## Overview
-Flink task type for executing Flink programs. For Flink nodes, the worker submits the task by using the Flink command `flink run`. See [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) for more details.
+Flink task type, used to execute Flink programs. For Flink nodes:
+
+(1) When the program type is Java, Scala or Python, the worker submits the task `flink run` using the Flink command. See [flink cli](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/) for more details.
+
+(2) When the program type is Sql, the worker submits the task `sql-client.sh` using Flink Sql. See [flink sql client](https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/) for more details.
Review Comment:
I'm sorry, I forgot. Wait a minute, I'll fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org