You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/06/22 03:10:07 UTC
[GitHub] [incubator-doris] wuyunfeng opened a new pull request #3924: [Doris On ES][Docs] update document for best practices
wuyunfeng opened a new pull request #3924:
URL: https://github.com/apache/incubator-doris/pull/3924
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] wutiangan commented on a change in pull request #3924: [Doris On ES][Docs] update document for best practices
Posted by GitBox <gi...@apache.org>.
wutiangan commented on a change in pull request #3924:
URL: https://github.com/apache/incubator-doris/pull/3924#discussion_r443301501
##########
File path: docs/zh-CN/extending-doris/doris-on-es.md
##########
@@ -449,6 +449,65 @@ select * from es_table where esquery(k4, ' {
4. Doris计算完结果后，返回给用户
+## 最佳实践
+
+### 时间类型字段使用建议
+
+在ES中，时间类型的字段使用十分灵活，但是在Doris On ES中如果对时间类型字段的类型设置不当，则会造成过滤条件无法下推
+
+创建索引时对时间类型格式的设置做最大程度的格式兼容:
+
+```
+ "dt": {
+ "type": "date",
+ "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
+ }
+```
+
+在Doris中建立该字段时建议设置为`date`或`datetime`,也可以设置为`varchar`类型, 使用如下SQL语句都可以直接将过滤条件下推至ES：
Review comment:
能否设置成char？
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] blackfox1983 commented on pull request #3924: [Doris On ES][Docs] update document for best practices
Posted by GitBox <gi...@apache.org>.
blackfox1983 commented on pull request #3924:
URL: https://github.com/apache/incubator-doris/pull/3924#issuecomment-647251044
lgtm
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] wutiangan commented on a change in pull request #3924: [Doris On ES][Docs] update document for best practices
Posted by GitBox <gi...@apache.org>.
wutiangan commented on a change in pull request #3924:
URL: https://github.com/apache/incubator-doris/pull/3924#discussion_r443301293
##########
File path: docs/zh-CN/extending-doris/doris-on-es.md
##########
@@ -449,6 +449,65 @@ select * from es_table where esquery(k4, ' {
4. Doris计算完结果后，返回给用户
+## 最佳实践
+
+### 时间类型字段使用建议
+
+在ES中，时间类型的字段使用十分灵活，但是在Doris On ES中如果对时间类型字段的类型设置不当，则会造成过滤条件无法下推
+
+创建索引时对时间类型格式的设置做最大程度的格式兼容:
+
+```
+ "dt": {
+ "type": "date",
+ "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
+ }
+```
+
+在Doris中建立该字段时建议设置为`date`或`datetime`,也可以设置为`varchar`类型, 使用如下SQL语句都可以直接将过滤条件下推至ES：
+
+```
+select * from doe where k2 > '2020-06-21';
+
+select * from doe where k2 < '2020-06-21 12:00:00';
+
+select * from doe where k2 < 1593497011;
+
+select * from doe where k2 < now();
+
+select * from doe where k2 < date_format(now(), '%Y-%m-%d');
+```
+
+注意:
+
+在ES中如果不对时间类型的字段设置`format`, 默认的时间类型字段格式为
+
+```
+strict_date_optional_time||epoch_millis
+```
+
+### 获取ES元数据字段`_id`
+
+导入文档在不指定`_id`的情况下ES会给每个文档分配一个全局唯一的`_id`元, 用户也可以在导入时为文档指定一个含有特殊业务意义的`_id`; 如果需要在Doris On ES中获取该字段值，建表时可以增加类型为`varchar`的`_id`字段：
Review comment:
`_id`元？
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #3924: [Doris On ES][Docs] update document for best practices
Posted by GitBox <gi...@apache.org>.
morningman merged pull request #3924:
URL: https://github.com/apache/incubator-doris/pull/3924
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] wuyunfeng commented on a change in pull request #3924: [Doris On ES][Docs] update document for best practices
Posted by GitBox <gi...@apache.org>.
wuyunfeng commented on a change in pull request #3924:
URL: https://github.com/apache/incubator-doris/pull/3924#discussion_r443302664
##########
File path: docs/zh-CN/extending-doris/doris-on-es.md
##########
@@ -449,6 +449,65 @@ select * from es_table where esquery(k4, ' {
4. Doris计算完结果后，返回给用户
+## 最佳实践
+
+### 时间类型字段使用建议
+
+在ES中，时间类型的字段使用十分灵活，但是在Doris On ES中如果对时间类型字段的类型设置不当，则会造成过滤条件无法下推
+
+创建索引时对时间类型格式的设置做最大程度的格式兼容:
+
+```
+ "dt": {
+ "type": "date",
+ "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
+ }
+```
+
+在Doris中建立该字段时建议设置为`date`或`datetime`,也可以设置为`varchar`类型, 使用如下SQL语句都可以直接将过滤条件下推至ES：
Review comment:
不建议用char，因为用户如果想那_id往往都是自己设置的，自己设置的ID长度大部分情况下可能并不一致
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org