You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@linkis.apache.org by "fjibj (via GitHub)" <gi...@apache.org> on 2023/02/13 03:02:24 UTC

[GitHub] [linkis] fjibj opened a new issue, #4208: [Bug] StartupParams has lost in CodeLogicalUnitExecTask.toRequestTask method

fjibj opened a new issue, #4208:
URL: https://github.com/apache/linkis/issues/4208

   ### Search before asking
   
   - [X] I searched the [issues](https://github.com/apache/linkis/issues) and found no similar issues.
   
   
   ### Linkis Component
   
   linkis-engineconnn-plugin, linkis-orchestrator
   
   ### Steps to reproduce
   
   Step 1:
   Add some attributes such as "jdbc.url" for workflow JDBC node, just insert some key in dss_workflow_node_ui table. You can see a new input appear in Attribute information of jdbc. So you can get the request json include "jdbc.url" in startup params.
   
   `params: {variable: {}, configuration: {special: {}, runtime: {,…}, startup: {ReuseEngine: "true",…}}}
   configuration: {special: {}, runtime: {,…}, startup: {ReuseEngine: "true",…}}
   runtime: {,…}
   contextID: "{"type":"HAWorkFlowContextID","value":"{\"instance\":null,\"backupInstance\":null,\"user\":\"root\",\"workspace\":\"bdapWorkspace\",\"project\":\"test22\",\"flow\":\"t22flow1\",\"contextId\":\"8-8--cs_1_devcs_1_dev20\",\"version\":\"v000001\",\"env\":\"BDAP_DEV\"}"}"
   nodeName: "jdbc_3011"
   special: {}
   startup: {ReuseEngine: "true",…}
   ReuseEngine: "true"
   jdbc.url: "jdbc:mysql://xx.xx.xx.xx:yyyy/mysql"
   variable: {}`
   
   
   Step 2:
   But you can't get the value of "jdbc.url" params from the properties of engineExecutorContext in EngineExecutor classes , such as JDBCEngineConnExecutor.scala, So, it must lost during the transmission process.
   
   ### Expected behavior
   
   I expect to get the value of startup params in the properties of engineExecutorContext in any EngineExecutor classes, or give me some other method to get them,maybe from JobHistory.
   
   But in my opinion, it is a public problem that needs to be fixed. I checked the code and found it lost in CodeLogicalUnitExecTask.toRequestTask method that passed a lot of parames except startup.
   
   I added some code in CodeLogicalUnitExecTask.toRequestTask method as an interim proposal
   `// add by fangjin 20230210
       if (null != getParams.getStartupParams) {
         requestTask.getProperties.putAll(getParams.getStartupParams.getConfigurationMap)
       }
   //`
   
   
   
   ### Your environment
   
   <!-- Please describe the linkis version you are using and basic environment information -->
   <!-- 请描述您使用的linkis版本和基本环境信息 -->
   - Linkis version used: 1.3.1
   - Environment name and version:
       - hadoop-3.2.1
       - hive-3.1.2
       - spark-2.4.6
       - scala-2.11.12
       - jdk 1.8.0_202
       - ....
   
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] Yes I am willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org

[GitHub] [linkis] github-actions[bot] commented on issue #4208: [Bug] StartupParams has lost in CodeLogicalUnitExecTask.toRequestTask method

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4208:
URL: https://github.com/apache/linkis/issues/4208#issuecomment-1427263441

   ## :blush:  Welcome to the Apache Linkis community!!
   We are glad that you are contributing by opening this issue.
   
   Please make sure to include all the relevant context.
   We will be here shortly.
   
   If you are interested in contributing to our website project, please let us know!
   You can check out our contributing guide on
    :point_right:  [How to Participate in Project Contribution](https://linkis.apache.org/community/how-to-contribute).
   
   
   ### Community
   
   |WeChat Assistant|WeChat Public Account|
   |-|-|
   |<img src="https://linkis.apache.org/Images/wedatasphere_contact_01.png" width="128"/>|<img src="https://linkis.apache.org/Images/gzh_01.png" width="128"/>|
   
   
   ### Mailing Lists
   |Name|Description|Subscribe|Unsubscribe|Archive|
   |:-----|:--------|:------|:-------|:-----|
   | [dev@linkis.apache.org](mailto:dev@linkis.apache.org) | community activity information | [subscribe](mailto:dev-subscribe@linkis.apache.org) | [unsubscribe](mailto:dev-unsubscribe@linkis.apache.org) | [archive](http://mail-archives.apache.org/mod_mbox/linkis-dev) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org

[GitHub] [linkis] casionone closed issue #4208: [Bug] StartupParams has lost in CodeLogicalUnitExecTask.toRequestTask method

Posted by "casionone (via GitHub)" <gi...@apache.org>.
casionone closed issue #4208: [Bug] StartupParams has lost in CodeLogicalUnitExecTask.toRequestTask method
URL: https://github.com/apache/linkis/issues/4208


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org

[GitHub] [linkis] casionone commented on issue #4208: [Bug] StartupParams has lost in CodeLogicalUnitExecTask.toRequestTask method

Posted by "casionone (via GitHub)" <gi...@apache.org>.
casionone commented on issue #4208:
URL: https://github.com/apache/linkis/issues/4208#issuecomment-1445281289

   I will close this issue. You can reopen if necessary


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org

[GitHub] [linkis] peacewong commented on issue #4208: [Bug] StartupParams has lost in CodeLogicalUnitExecTask.toRequestTask method

Posted by "peacewong (via GitHub)" <gi...@apache.org>.
peacewong commented on issue #4208:
URL: https://github.com/apache/linkis/issues/4208#issuecomment-1434050524

   Thank you for your explanation, this should be a DSS bug. Because StartUp should be the parameter used when starting Ec. You can refer to: org.apache.linkis.orchestrator.computation.execute.DefaultCodeExecTaskExecutorManager#createMarkReq
   The URL of JDBC should be put into Runtime parmas.
   But I think it can be compatible here, but in order to prevent parameter duplication, it is recommended to put startUP first and then put runtime


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org