You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@linkis.apache.org by GitBox <gi...@apache.org> on 2022/03/31 10:03:44 UTC
[GitHub] [incubator-linkis] CCweixiao opened a new pull request #1848: Script custom variable run type and suffix constraint configuration.
CCweixiao opened a new pull request #1848:
URL: https://github.com/apache/incubator-linkis/pull/1848
### What is the purpose of the change
引擎类型、脚本类型自定义后，脚本中的变量解析异常，需要在源码中修改几个脚本，基于此，需要把此类配置抽取出来作为配置文件中的一个配置项。https://github.com/apache/incubator-linkis/issues/1751
After the engine type and script type are customized, the variable parsing in the script is abnormal, and several scripts need to be modified in the source code. Based on this, such configuration needs to be extracted as a configuration item in the configuration file. https://github.com/apache/incubator-linkis/issues/1751
### Brief change log
脚本类型与运行类型对应关系在org.apache.linkis.common.utils.CodeAndRunTypeUtils维护，
val CODE_TYPE_AND_RUN_TYPE_RELATION = CommonVars("wds.linkis.codeType.runType.relation", "sql=>sql|hql|jdbc|hive|psql|fql,python=>python|py|pyspark,java=>java,scala=>scala,shell=>sh|shell")
工具方法
getCodeTypeAndRunTypeRelationMap
getRunTypeAndCodeTypeRelationMap
测试用例，已全部通过
org.apache.linkis.common.utils.CodeAndRunTypeUtilsTest
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org
[GitHub] [incubator-linkis] peacewong commented on a change in pull request #1848: Script custom variable run type and suffix constraint configuration.
Posted by GitBox <gi...@apache.org>.
peacewong commented on a change in pull request #1848:
URL: https://github.com/apache/incubator-linkis/pull/1848#discussion_r839451109
##########
File path: linkis-computation-governance/linkis-computation-governance-common/src/main/scala/org/apache/linkis/governance/common/paser/CodeParser.scala
##########
@@ -300,12 +302,18 @@ object CodeType extends Enumeration {
type CodeType = Value
val Python, SQL, Scala, Shell, Other, Remain, JSON = Value
- def getType(codeType: String): CodeType = codeType.toLowerCase() match {
- case "python" | "pyspark" | "py" => Python
- case "sql" | "hql" | "psql" => SQL
- case "scala" => Scala
- case "shell" | "sh" => Shell
- case _ => Other
+ def getType(codeType: String): CodeType = {
+ val runTypeAndCodeTypeRelationMap: Map[String, String] = CodeAndRunTypeUtils.getRunTypeAndCodeTypeRelationMap
+ if (runTypeAndCodeTypeRelationMap.isEmpty) return Other
Review comment:
Is this better?
`if (runTypeAndCodeTypeRelationMap.isEmpty || !runTypeAndCodeTypeRelationMap.contains(codeType) ) return Other`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org
[GitHub] [incubator-linkis] peacewong commented on a change in pull request #1848: Script custom variable run type and suffix constraint configuration.
Posted by GitBox <gi...@apache.org>.
peacewong commented on a change in pull request #1848:
URL: https://github.com/apache/incubator-linkis/pull/1848#discussion_r839451109
##########
File path: linkis-computation-governance/linkis-computation-governance-common/src/main/scala/org/apache/linkis/governance/common/paser/CodeParser.scala
##########
@@ -300,12 +302,18 @@ object CodeType extends Enumeration {
type CodeType = Value
val Python, SQL, Scala, Shell, Other, Remain, JSON = Value
- def getType(codeType: String): CodeType = codeType.toLowerCase() match {
- case "python" | "pyspark" | "py" => Python
- case "sql" | "hql" | "psql" => SQL
- case "scala" => Scala
- case "shell" | "sh" => Shell
- case _ => Other
+ def getType(codeType: String): CodeType = {
+ val runTypeAndCodeTypeRelationMap: Map[String, String] = CodeAndRunTypeUtils.getRunTypeAndCodeTypeRelationMap
+ if (runTypeAndCodeTypeRelationMap.isEmpty) return Other
Review comment:
Is this better?
`if (runTypeAndCodeTypeRelationMap.isEmpty || runTypeAndCodeTypeRelationMap.contains(codeType) ) return Other`
##########
File path: linkis-commons/linkis-common/src/main/scala/org/apache/linkis/common/utils/CodeAndRunTypeUtils.scala
##########
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.linkis.common.utils
+
+import org.apache.linkis.common.conf.CommonVars
+
+object CodeAndRunTypeUtils {
+ private val CONF_LOCK = new Object()
+
+ val CODE_TYPE_AND_RUN_TYPE_RELATION = CommonVars("wds.linkis.codeType.runType.relation", "sql=>sql|hql|jdbc|hive|psql|fql,python=>python|py|pyspark,java=>java,scala=>scala,shell=>sh|shell")
+
+ val RUN_TYPE_SQL = "sql"
+ val RUN_TYPE_PYTHON = "python"
+ val RUN_TYPE_JAVA = "java"
+ val RUN_TYPE_SCALA = "scala"
+ val RUN_TYPE_SHELL = "shell"
+
+ private var codeTypeAndRunTypeRelationMap: Map[String, List[String]] = null
+
+ private def codeTypeAndRunTypeRelationMapParser(configV: String): Map[String, List[String]] = {
+ val confDelimiter = ","
+ if (configV == null || "".equals(configV)) Map()
+ else configV.split(confDelimiter).filter(x => x != null && !"".equals(x)).map(x => {
+ val confArr = x.split("=>")
+ if (confArr.length == 2) (confArr(0), for (x <- confArr(1).split("\\|").toList) yield x.trim) else null
+ }).filter(x => x != null).toMap
+ }
+
+ def getCodeTypeAndRunTypeRelationMap: Map[String, List[String]] = CONF_LOCK.synchronized {
+ if (null == codeTypeAndRunTypeRelationMap) {
Review comment:
The lock is recommended after the if, which will cause a synchronous wait in front of the entire method
锁建议在if后面，在整个方法前面会导致同步等待
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org
[GitHub] [incubator-linkis] casionone merged pull request #1848: Script custom variable run type and suffix constraint configuration.
Posted by GitBox <gi...@apache.org>.
casionone merged pull request #1848:
URL: https://github.com/apache/incubator-linkis/pull/1848
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org