You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@linkis.apache.org by "casionone (via GitHub)" <gi...@apache.org> on 2023/03/06 02:29:26 UTC
[GitHub] [linkis] casionone opened a new issue, #4302: Optimize scriptis code with @set module distinguish between scriptContent/metadata by annotations
casionone opened a new issue, #4302:
URL: https://github.com/apache/linkis/issues/4302
## Description
Edit variable entry Edit page>Configuration>Configuration panel entry Pass through the metadata.variable field when saving, and the hdfs file will be left after the backend parses
Requested interface input parameters when saving
```
{"fileName":"229cf765-6839-4c82-829d-1907c2ccf668.sql","scriptContent":"select ${qq};\n--@set test=123\nselect ${test};\n-- @set qq=222\nselect ${qq};\n","projectName":"test1122a1","metadata":{"variable":{"qq":"123"},"configuration":{"special ":{},"runtime":{},"startup":{}}},"resourceId":"c9755cad-619b-4c1c-9204-cc4bb9836194"}
```
Execution result The variable that will be parsed before the code is executed, so the value of the variable behind takes effect
```
222 qq=222
123 test=123
222 qq=222
```
The code storage of the actual hdfs file, the panel value of the sidebar is always appended to the front of the file through the --@set method
```
[hadoop~]$ hdfs dfs -cat /apps-data/hadoop/bml/20230208/c9755cad-619b-4c1c-9204-cc4bb9836194_v000005
--@set qq=123
select ${qq};
--@set test=123
select ${test};
--@set qq=222
select ${qq};
```
Re-open call interface
/api/rest_j/v1/filesystem/openScriptFromBML?fileName=229cf765-6839-4c82-829d-1907c2ccf668.sql&resourceId=c9755cad-619b-4c1c-9204-cc4bb9836194&version=v000008&creator=&projectNamea=1test112
the
The hdfs file content is parsed into two parts scriptContent and metadata
```
{"method":"/api/filesystem/openScriptFromBML","status":0,"message":"OK","data":{"scriptContent":"select ${qq};\n--@set test=123\nselect ${test};\n--@set qq=222\nselect ${qq};\n","metadata":{"variable":{"qq":"123","tt ":"456"}}}}
```
How is metadata.variable obtained? The front of the code (beginning with --@set to the end of non--@set) --@set is parsed as metadata.variable and other data is still retained in the scriptContent code
the
**Solutions**
- When saving: the value of metadata.variable (the value set in the side panel) and the content of scriptContent are spliced and separated by annotations, such as sql "--" and saved to the file
- When querying: Call the interface to query data/api/rest_j/v1/filesystem/openScriptFromBML When parsing the file, use the comment delimiter to divide the script content into two parts, parse the value in front of the comment into metadata.variable value and echo it in the variable panel,
- The value after the comment is used as scriptContent
------
编辑变量的入口 编辑页面>配置 >配置面板入口 保存时 通过metadata.variable字段传递，后端解析后落hdfs 文件
保存时 请求的接口入参
```
{"fileName":"229cf765-6839-4c82-829d-1907c2ccf668.sql","scriptContent":"select ${qq};\n--@set test=123\nselect ${test};\n--@set qq=222\nselect ${qq};\n","projectName":"test1122a1","metadata":{"variable":{"qq":"123"},"configuration":{"special":{},"runtime":{},"startup":{}}},"resourceId":"c9755cad-619b-4c1c-9204-cc4bb9836194"}
```
执行结果 代码执行前会先解析的变量 所以是位于后面的变量值生效
```
222 qq=222
123 test=123
222 qq=222
```
实际hdfs文件的代码存储，侧边栏的面板值始终通过--@set方式 在文件最前面追加
```
[hadoop~]$ hdfs dfs -cat /apps-data/hadoop/bml/20230208/c9755cad-619b-4c1c-9204-cc4bb9836194_v000005
--@set qq=123
select ${qq};
--@set test=123
select ${test};
--@set qq=222
select ${qq};
```
重新打开 调用接口
/api/rest_j/v1/filesystem/openScriptFromBML?fileName=229cf765-6839-4c82-829d-1907c2ccf668.sql&resourceId=c9755cad-619b-4c1c-9204-cc4bb9836194&version=v000008&creator=&projectName=test1122a1
hdfs文件内容 被解析为两部分 scriptContent 和metadata
```
{"method":"/api/filesystem/openScriptFromBML","status":0,"message":"OK","data":{"scriptContent":"select ${qq};\n--@set test=123\nselect ${test};\n--@set qq=222\nselect ${qq};\n","metadata":{"variable":{"qq":"123","tt":"456"}}}}
```
metadata.variable 如何获取的？ 代码中最前面的(--@set开始到 非--@set 结束) --@set 被解析为 metadata.variable 其他数据依旧被保留在scriptContent 代码中
**解决思路**
- 保存时： metadata.variable的值（侧边面板设置的值） 和 scriptContent 内容进行拼接，通过注解符进行分隔 如sql “--“ 保存到文件中
- 查询时： 调用接口查询数据/api/rest_j/v1/filesystem/openScriptFromBML 解析文件时 通过注解分隔符 将脚本内容一份为二 注解符前面的 解析为metadata.variable值 在变量面板中回显，
- 注解符后面的值 作为scriptContent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [linkis] jacktao007 closed issue #4302: Optimize scriptis code with @set module distinguish between scriptContent/metadata by annotations
Posted by "jacktao007 (via GitHub)" <gi...@apache.org>.
jacktao007 closed issue #4302: Optimize scriptis code with @set module distinguish between scriptContent/metadata by annotations
URL: https://github.com/apache/linkis/issues/4302
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [linkis] github-actions[bot] commented on issue #4302: Optimize scriptis code with @set module distinguish between scriptContent/metadata by annotations
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4302:
URL: https://github.com/apache/linkis/issues/4302#issuecomment-1455338164
## :blush: Welcome to the Apache Linkis community!!
We are glad that you are contributing by opening this issue.
Please make sure to include all the relevant context.
We will be here shortly.
If you are interested in contributing to our website project, please let us know!
You can check out our contributing guide on
:point_right: [How to Participate in Project Contribution](https://linkis.apache.org/community/how-to-contribute).
### Community
|WeChat Assistant|WeChat Public Account|
|-|-|
|<img src="https://linkis.apache.org/Images/wedatasphere_contact_01.png" width="128"/>|<img src="https://linkis.apache.org/Images/gzh_01.png" width="128"/>|
### Mailing Lists
|Name|Description|Subscribe|Unsubscribe|Archive|
|:-----|:--------|:------|:-------|:-----|
| [dev@linkis.apache.org](mailto:dev@linkis.apache.org) | community activity information | [subscribe](mailto:dev-subscribe@linkis.apache.org) | [unsubscribe](mailto:dev-unsubscribe@linkis.apache.org) | [archive](http://mail-archives.apache.org/mod_mbox/linkis-dev) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org