[Discuss] [Linkis-PS] Replace the resultSet of Linkis defined 'dolphin' format to parquet.

[尹强 <en...@apache.org>]

Changing the current result set of Linkis from the custom Dolphin format to
Parquet for storage will greatly improve the performance of Linkis's result
set transmission and secondary processing calculations, especially for the
high-level operations of the Scriptis result set display module. : Sorting,
filtering, etc., as well as the CS context capabilities of the DSS
workflow, including the later Linkis Orchestrator to achieve orchestration
capabilities, and heterogeneous data sources hybrid computing, can greatly
improve performance.
What do you think, is there a partner who claims the demand?

-----------------------------------------------

将Linkis现在的结果集由自定义的Dolphin格式改为Parquet进行存储，将为Linkis的结果集传输、二次加工计算起到巨大的性能提升，特别是对Scriptis结果集展示模块的高阶操作如：排序、过滤等，以及DSS工作流的CS上下文能力，包括后面Linkis
Orchestrator实现编排能力，异构数据源混合计算时，都能非常大地提升性能。
大家怎么看，是否有认领该需求的伙伴？

Re: [Discuss] [Linkis-PS] Replace the resultSet of Linkis defined 'dolphin' format to parquet.

[Zhen Wang <wa...@apache.org>]

Hi Qiang Yin,
At present, the dolphin storage format can store multiple types of result
sets such as table, txt, etc., while the parquet format seems to only store
table type results. Do you have any suggestions for this?

尹强 <en...@apache.org> 于2021年12月30日周四 14:11写道：

> Changing the current result set of Linkis from the custom Dolphin format to
> Parquet for storage will greatly improve the performance of Linkis's result
> set transmission and secondary processing calculations, especially for the
> high-level operations of the Scriptis result set display module. : Sorting,
> filtering, etc., as well as the CS context capabilities of the DSS
> workflow, including the later Linkis Orchestrator to achieve orchestration
> capabilities, and heterogeneous data sources hybrid computing, can greatly
> improve performance.
> What do you think, is there a partner who claims the demand?
>
> -----------------------------------------------
>
>
> 将Linkis现在的结果集由自定义的Dolphin格式改为Parquet进行存储，将为Linkis的结果集传输、二次加工计算起到巨大的性能提升，特别是对Scriptis结果集展示模块的高阶操作如：排序、过滤等，以及DSS工作流的CS上下文能力，包括后面Linkis
> Orchestrator实现编排能力，异构数据源混合计算时，都能非常大地提升性能。
> 大家怎么看，是否有认领该需求的伙伴？
>