You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@linkis.apache.org by 康悦Rita <ri...@163.com> on 2022/09/21 02:49:41 UTC
[DISCUSS]Doris Engine related discussion
Dear:
Wechat group "Apache Linkis community Development group "chat records are as follows: 微信群"Apache Linkis 社区开发群"的聊天记录如下:
————— 2022-9-15 —————
Lentils also
Did you guys write the Doris engine
Zhi-yuan liu 15:33 the
I thought Doris was mysql driven
Lentils 15:56
Yes.
Lentils 15:56
I've already written that the Doris engine is the mysql driver
Lentils 15:57
The Doris engine is divided into two parts: 1. Synchronization and 2. Computing
Zhi-yuan liu 16:00
@Lentil He does a lot of syncing.
Lentils 16:06
I've already implemented Doris synchronization in FlinkX
Lentils 16:07
I've tested that the Doris engine is computationally slow, especially for large amounts of data that can't be executed locally
Zhi-yuan liu 16:07
What version of Doris are you using
Lentils 16:07
doris1.1
Zhi-yuan liu 16:07
It is what respect is slower ah, am I used here still OK ah
Zhi-yuan liu 16:08
Query with table or calculate what data?
Lentils 16:08
For example, my current implementation of the Doris computing engine has the mysql driver at the bottom of it and if I do insert select or something like that it's slow
Zhi-yuan liu 16:09
How much data? What kind of model? That should be pretty quick
Zhi-yuan liu"
He's actually doing all the asynchronous tasks
Lentils, when
Duplicate
Zhi-yuan liu affliction
Want to use as a mysql table? There is still no need to remove weights and precompute
Zhi-yuan liu concealed
@Lentil, go to the Doris website and just wechat them. Go in and ask the big guy
Lentils 16:17
I give them big issues
Lentils 16:18
The Doris audit log plug-in and the BUG just got back to me today
Lentils 16:18
By the time you get back to me, I've already solved the problem
Lentil is better
I'm just doing pre-technical research and then I start breaking things up and I can't send them out
Zhi-yuan liu is better
It would be better if you went to the group to ask, they seem to be very busy distributing packets a while ago
Zhi-yuan liu is better
Most of them are answered by selectDb and Baidu
Lentils"
thank you
Zosimer 16:23
Slower than Hive?
Lentils for
It's still faster than Hive. I'm still not familiar with Doris executing SQL
Zhi-yuan liu for
An upgraded version of Impala. My initial feeling was
Zhi-yuan liu 16:27
Create AGG models for real-time calculations
Zosimer 16:27
Have you tried the single form yet?
Zosimer 16:28
Tried Doris more table than CK block
Zhi-yuan liu"
Ck's multi table itself is relatively slow
Zosimer 16:29
Ck multiple tables don't work
Zhi-yuan liu"
Doris has a Colocation Join optimization, and it would be a lot faster to optimize if you could use that when you're building tables
扁豆 15:14
各位大佬 doris引擎写了么
刘志远 15:33
doris不是mysql驱动么
扁豆 15:56
是的
扁豆 15:56
我已经写了doris引擎 就是mysql驱动
扁豆 15:57
doris引擎分为两部分：1、同步 2、计算
刘志远 16:00
@扁豆 他同步方式还挺多的。
扁豆 16:06
这个我已经在flinkx里面实现doris同步
扁豆 16:07
我测试过doris引擎计算很慢 尤其是大数据量 在本地执行 还不能满足要求
刘志远 16:07
你用的doris是什么版本的
扁豆 16:07
doris1.1
刘志远 16:07
是什么方面比较慢呀，我这里用着还可以呀
刘志远 16:08
连表查询还是计算什么数据？
扁豆 16:08
比如 我现在实现的doris计算引擎 底层就是mysql驱动 如果我执行insert select之类语句 是比较慢的
刘志远 16:09
多少数据量呀，用的是什么模型？应该挺快的呀
刘志远 16:10
他其实都是做的异步任务
扁豆 16:11
Duplicate
刘志远 16:12
是想当作一个mysql表去用？还是没有去重和预计算的需求么
刘志远 16:17
@扁豆 你去doris官网直接加他们微信，进去问问他们大佬吧
扁豆 16:17
我都给他们大佬提了issue
扁豆 16:18
doris审计日志插件还有BUG 今天才回复我的
扁豆 16:18
回复我的时候 我已经解决了 没有微众大佬及时
扁豆 16:19
我现在只是做前期技术预研 预研完了就开始分解任务 发不出去
刘志远 16:19
你去群里问问会好些，前阵子他们发包好像都挺忙的
刘志远 16:19
大部分都是selectDb和百度的去回答
扁豆 16:22
谢谢
Zosimer 16:23
比hive还慢？
扁豆 16:26
比hive还是要快的 我现在还不熟悉doris执行sql机制
刘志远 16:26
impala的升级版本，我最初的感觉就是
刘志远 16:27
创建agg模型做实时运算
Zosimer 16:27
单表试过没？
Zosimer 16:28
试过doris多表比ck块
刘志远 16:29
ck的多表本身就比较慢吧
Zosimer 16:29
ck多表不行
刘志远 16:29
doris有一个Colocation Join的优化方式，如果建表的时候能用这个优化下会快很多
--
Best Regards
------
康悦 ritakang
GitHub：Ritakang0451
E-mail：rita0451@163.com