You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2023/01/12 02:29:33 UTC
[GitHub] [doris] xiebanggui777 opened a new issue, #15847: Can the planning of multi-table materialized view realize the view function of view and directly use the materialized view name for sql query? 多表物化视图的规划能否实现视图的视图功能和直接使用物化视图名称进行sql查询? [Feature]
xiebanggui777 opened a new issue, #15847:
URL: https://github.com/apache/doris/issues/15847
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Description
I see that the later version of doris has the function of multi-table materialized view, which is a very practical function. I have two business scenario requirements to see if doris will consider adding them later.
①View of the view
Create a materialized view on the materialized view that has been created
②Mask the details of the sql statement and query directly through the materialized view name
Users in the use of materialized views without through complex SQL statements, query directly by the name of the materialized views, shielding the SQL statement details, for example: select * from materialized_view_name
我看到doris的后期版本规划有多表物化视图的功能，这是非常实用的功能，我有2个业务场景需求看看doris后期是否会考虑加入在里面。
①视图的视图
在已创建的物化视图上再进行创建物化视图
②屏蔽sql语句细节，直接通过物化视图名称进行查询
用户在使用物化视图时无需通过复杂的sql语句，通过物化视图的名称直接查询，屏蔽sql语句细节，例如：select * from 物化视图名称
### Use case
View the view function and by name query function of materialized views, these two functions to improve the real-time of scenario Doris in data warehouse and reduce the dependence on other third-party components:
For example, in the data warehouse ODS level (source data level) table create multi-table materialized view to generate DWD level (data detail level) table, through the name of the materialized view directly sql query can query to the corresponding DWD table, and then on the basis of the DWD table to create materialized view to generate DWS level (data summary level) table.
like ODS - > DWD (view) - > DWS (view) - > ADS (view)
This has the advantage of three:
(1) for data integration and synchronization, users only need to focus on ODS level data real-time and integrity.
(2) because of other data levels were calculated by means of materialized views logic of writing, to improve the real-time performance, and no need for third-party scheduling components is required for script scheduling between data levels.
(3) the data synchronization in ods levels , other data levels synchronization update and change, Improve the ability of Doris real-time data warehouse.
视图的视图功能和通过名称查询物化视图功能，这2个功能有利于提高doris在数据仓库场景下的实时性和减少对其他第三方组件的依赖：
例如在数据仓库ODS层级(源数据层级)直接创建多表物化视图生成DWD层级(数据明细层级)的表格，通过该物化视图的名称直接sql查询就可以查询到对应的DWD表格，然后在DWD表格的基础上再进行创建物化视图生成DWS层级(数据汇总层级)表格。
ODS->DWD(视图)->DWS(视图)->ADS(视图)
这样做的好处有3个：
①关于数据集成和同步方面，用户只需要关注ods层的数据实时性和完整性。
②由于其他层级通过物化视图的方式进行计算逻辑的编写，提高了实时性，且无需第三方调度组件进行数据层级之间的脚本调度。
③数据同步进来ods层，其他层级数据也同步进行更新和变化，提高了doris实时数据仓库方面的能力。
### Related issues
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Accelarator commented on issue #15847: Can the planning of multi-table materialized view realize the view function of view and directly use the materialized view name for sql query? 多表物化视图的规划能否实现视图的视图功能和直接使用物化视图名称进行sql查询? [Feature]
Posted by GitBox <gi...@apache.org>.
Accelarator commented on issue #15847:
URL: https://github.com/apache/doris/issues/15847#issuecomment-1379916468
附议，我们目前就是通过秒级别的调度来实现实时数仓不同层级之间ETL的，该需求也会极大方便我们，希望能尽快实现
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] kpfly commented on issue #15847: Can the planning of multi-table materialized view realize the view function of view and directly use the materialized view name for sql query? 多表物化视图的规划能否实现视图的视图功能和直接使用物化视图名称进行sql查询? [Feature]
Posted by GitBox <gi...@apache.org>.
kpfly commented on issue #15847:
URL: https://github.com/apache/doris/issues/15847#issuecomment-1379747585
Yes，we will support these features definitely.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xiebanggui777 commented on issue #15847: Can the planning of multi-table materialized view realize the view function of view and directly use the materialized view name for sql query? 多表物化视图的规划能否实现视图的视图功能和直接使用物化视图名称进行sql查询? [Feature]
Posted by GitBox <gi...@apache.org>.
xiebanggui777 commented on issue #15847:
URL: https://github.com/apache/doris/issues/15847#issuecomment-1379924370
如果能实现视图的视图功能，且复杂sql的物化视图性能上不输Flink的话，很大一部分实时计算任务会直接使用Doris来实现。哈哈哈
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org