You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "weizhengte (via GitHub)" <gi...@apache.org> on 2023/05/05 15:39:26 UTC
[GitHub] [doris] weizhengte opened a new pull request, #19323: [Doc](statistics) add statistics document
weizhengte opened a new pull request, #19323:
URL: https://github.com/apache/doris/pull/19323
# Proposed changes
Issue Number: close #xxx
## Problem summary
Describe your changes.
## Checklist(Required)
* [ ] Does it affect the original behavior
* [ ] Has unit tests been added
* [ ] Has document been added or modified
* [ ] Does it need to update dependencies
* [ ] Is this PR support rollback (If NO, please explain WHY)
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on PR #19323:
URL: https://github.com/apache/doris/pull/19323#issuecomment-1537030631
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow merged pull request #19323: [Doc](statistics) add statistics document
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow merged PR #19323:
URL: https://github.com/apache/doris/pull/19323
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19323: [Doc](statistics) add statistics document
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19323:
URL: https://github.com/apache/doris/pull/19323#issuecomment-1545282609
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186657307
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
Review Comment:
We're not going to store table-level stats at this point, but we're going to use them to help automate the collection of statistics, such as size or count, which are taken directly from the metadata, but are listed here for `show table stats`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186657725
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
+| `healthy` | 表的健康度 |
+| `update_time` | 最近更新的时间 |
+| `last_analyze_time` | 上次收集统计信息的时间 |
+
+> 表的健康度：表示表统计信息的健康程度。当 `update_rows` 大于等于 `row_count` 时，健康度为 0；当 `update_rows` 小于 `row_count` 时，健康度为 `100 * (1 - update_rows / row_count)` 。
+
+列统计信息：
+
+| 信息 | 描述 |
+| :------------ | :------------------------- |
+| `row_count` | 列的总行数 |
Review Comment:
yes, the row_count of column statistics should be optimized and removed in the future
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on PR #19323:
URL: https://github.com/apache/doris/pull/19323#issuecomment-1545349346
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on PR #19323:
URL: https://github.com/apache/doris/pull/19323#issuecomment-1536566539
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186656504
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
+| `healthy` | 表的健康度 |
+| `update_time` | 最近更新的时间 |
+| `last_analyze_time` | 上次收集统计信息的时间 |
Review Comment:
Take the latest one for now, whether it's just one column or not. Think that analyze one column is a special case
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186655810
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
Review Comment:
Get the size of the table for display only
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "Kikyou1997 (via GitHub)" <gi...@apache.org>.
Kikyou1997 commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186654657
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
+| `healthy` | 表的健康度 |
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "Kikyou1997 (via GitHub)" <gi...@apache.org>.
Kikyou1997 commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186651997
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
+| `healthy` | 表的健康度 |
+| `update_time` | 最近更新的时间 |
+| `last_analyze_time` | 上次收集统计信息的时间 |
+
+> 表的健康度：表示表统计信息的健康程度。当 `update_rows` 大于等于 `row_count` 时，健康度为 0；当 `update_rows` 小于 `row_count` 时，健康度为 `100 * (1 - update_rows / row_count)` 。
+
+列统计信息：
+
+| 信息 | 描述 |
+| :------------ | :------------------------- |
+| `row_count` | 列的总行数 |
Review Comment:
After table level stats supported, row_count in col_stats is redandunt I think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186657307
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
Review Comment:
For now, only partial info(row count, anaalyzed time) are persisted in table-level stats, to use them to help automate the collection of statistics, but are listed here for `show table stats`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on PR #19323:
URL: https://github.com/apache/doris/pull/19323#issuecomment-1536560173
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "Kikyou1997 (via GitHub)" <gi...@apache.org>.
Kikyou1997 commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186654708
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
Review Comment:
What is this field used for
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186657307
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
Review Comment:
For now, not store table-level stats, but to use them to help automate the collection of statistics, such as size or count, which are taken directly from the metadata, but are listed here for `show table stats`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on PR #19323:
URL: https://github.com/apache/doris/pull/19323#issuecomment-1537025795
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "Kikyou1997 (via GitHub)" <gi...@apache.org>.
Kikyou1997 commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186655186
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
+| `healthy` | 表的健康度 |
+| `update_time` | 最近更新的时间 |
+| `last_analyze_time` | 上次收集统计信息的时间 |
+
+> 表的健康度：表示表统计信息的健康程度。当 `update_rows` 大于等于 `row_count` 时，健康度为 0；当 `update_rows` 小于 `row_count` 时，健康度为 `100 * (1 - update_rows / row_count)` 。
+
+列统计信息：
+
+| 信息 | 描述 |
+| :------------ | :------------------------- |
+| `row_count` | 列的总行数 |
+| `data_size` | 列的总⻓度（单位 byte） |
+| `avg_size_byte` | 列的平均⻓度（单位 bytes） |
+| `ndv` | 列 num distinct value |
+| `min` | 列最小值 |
+| `max` | 列最⼤值 |
+| `null_count` | 列 null 个数 |
+| `histogram` | 列直方图 |
+
+接下来将简单介绍其中出现的直方图等数据结构，以及详细介绍统计信息的收集和维护。
+
+## 直方图简介
+
+直方图（histogram）是一种用于描述数据分布情况的工具，它将数据根据大小分成若干个区间（桶），并使用简单的统计量来表示每个区间中数据的特征。是数据库中的一种重要的统计信息，可以描述列中的数据分布情况。直方图最典型的应用场景是通过估算查询谓词的选择率来帮助优化器选择最优的执行计划。
+
+在 Doris 中，会对每个表具体的列构建一个等高直方图（Equi-height Histogram）。直方图包括一系列桶， 其中每个桶的统计量包括桶的上下界、桶包含的元素数量、前面桶所有元素的数量、桶中不同值的个数。具体可以参考 SQL 函数 `histogram` 或者 `hist` 的使用说明。
+
+> 采用等高直方图的分桶方法，每个桶中的数值频次之和都应该接近于总行数的 `1/N`。但是，如果严格遵守等高原则进行分桶，会出现某些值落在桶的边界上的情况，导致同一个值出现在两个不同的桶中。这种情况会对选择率的估算造成干扰。因此，在实现中，Doris 对等高直方图的分桶方法进行了修改：如果将一个值加入到某个桶中导致该桶中的数据频次超过了总行数的 `1/N`，则根据哪种情况更接近 `1/N`，将该值放入该桶或下一个桶中。
+
+## 收集统计信息
+
+### 手动收集
+
+⽤户通过 `ANALYZE` 语句触发手动收集任务，根据提供的参数，收集指定的表或列的统计信息。
+
+列统计信息收集语法：
+
+```SQL
+ANALYZE [ SYNC ] TABLE table_name
+ [ (column_name [, ...]) ]
+ [ [ WITH SYNC ] [ WITH INCREMENTAL ] [ WITH SAMPLE PERCENT | ROWS ] [ WITH PERIOD ] ]
+ [ PROPERTIES ("key" = "value", ...) ];
+```
+
+列直方图收集语法：
+
+```SQL
+ANALYZE [ SYNC ] TABLE table_name
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186657725
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
+| `healthy` | 表的健康度 |
+| `update_time` | 最近更新的时间 |
+| `last_analyze_time` | 上次收集统计信息的时间 |
+
+> 表的健康度：表示表统计信息的健康程度。当 `update_rows` 大于等于 `row_count` 时，健康度为 0；当 `update_rows` 小于 `row_count` 时，健康度为 `100 * (1 - update_rows / row_count)` 。
+
+列统计信息：
+
+| 信息 | 描述 |
+| :------------ | :------------------------- |
+| `row_count` | 列的总行数 |
Review Comment:
The following explains
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19323: [Doc](statistics) add statistics document
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19323:
URL: https://github.com/apache/doris/pull/19323#issuecomment-1545282552
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "Kikyou1997 (via GitHub)" <gi...@apache.org>.
Kikyou1997 commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186653261
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
+| `healthy` | 表的健康度 |
+| `update_time` | 最近更新的时间 |
+| `last_analyze_time` | 上次收集统计信息的时间 |
Review Comment:
What' the definition of `last_analyze_time` of a table, since user could just analyze one column in the table
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "Kikyou1997 (via GitHub)" <gi...@apache.org>.
Kikyou1997 commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186654618
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
Review Comment:
Do we really need to persist this field?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on PR #19323:
URL: https://github.com/apache/doris/pull/19323#issuecomment-1545273615
> Document titles are best served with CBO statistics, and a document directory needs to be added to sidebar
This pr will not be changed, the work of catalogue classification is in progress
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186657307
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
Review Comment:
For now, only partial info(row_count, last_analyze_time) are persisted in table-level stats, to use them to help automate the collection of statistics, but are listed here for `show table stats`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186655706
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
+| `healthy` | 表的健康度 |
+| `update_time` | 最近更新的时间 |
+| `last_analyze_time` | 上次收集统计信息的时间 |
+
+> 表的健康度：表示表统计信息的健康程度。当 `update_rows` 大于等于 `row_count` 时，健康度为 0；当 `update_rows` 小于 `row_count` 时，健康度为 `100 * (1 - update_rows / row_count)` 。
+
+列统计信息：
+
+| 信息 | 描述 |
+| :------------ | :------------------------- |
+| `row_count` | 列的总行数 |
+| `data_size` | 列的总⻓度（单位 byte） |
+| `avg_size_byte` | 列的平均⻓度（单位 bytes） |
+| `ndv` | 列 num distinct value |
+| `min` | 列最小值 |
+| `max` | 列最⼤值 |
+| `null_count` | 列 null 个数 |
+| `histogram` | 列直方图 |
+
+接下来将简单介绍其中出现的直方图等数据结构，以及详细介绍统计信息的收集和维护。
+
+## 直方图简介
+
+直方图（histogram）是一种用于描述数据分布情况的工具，它将数据根据大小分成若干个区间（桶），并使用简单的统计量来表示每个区间中数据的特征。是数据库中的一种重要的统计信息，可以描述列中的数据分布情况。直方图最典型的应用场景是通过估算查询谓词的选择率来帮助优化器选择最优的执行计划。
+
+在 Doris 中，会对每个表具体的列构建一个等高直方图（Equi-height Histogram）。直方图包括一系列桶， 其中每个桶的统计量包括桶的上下界、桶包含的元素数量、前面桶所有元素的数量、桶中不同值的个数。具体可以参考 SQL 函数 `histogram` 或者 `hist` 的使用说明。
+
+> 采用等高直方图的分桶方法，每个桶中的数值频次之和都应该接近于总行数的 `1/N`。但是，如果严格遵守等高原则进行分桶，会出现某些值落在桶的边界上的情况，导致同一个值出现在两个不同的桶中。这种情况会对选择率的估算造成干扰。因此，在实现中，Doris 对等高直方图的分桶方法进行了修改：如果将一个值加入到某个桶中导致该桶中的数据频次超过了总行数的 `1/N`，则根据哪种情况更接近 `1/N`，将该值放入该桶或下一个桶中。
+
+## 收集统计信息
+
+### 手动收集
+
+⽤户通过 `ANALYZE` 语句触发手动收集任务，根据提供的参数，收集指定的表或列的统计信息。
+
+列统计信息收集语法：
+
+```SQL
+ANALYZE [ SYNC ] TABLE table_name
Review Comment:
This one is left over and will be removed later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186657307
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
Review Comment:
For now, only partial info is persisted in table-level stats, to use them to help automate the collection of statistics, but are listed here for `show table stats`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "Kikyou1997 (via GitHub)" <gi...@apache.org>.
Kikyou1997 commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186651997
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
+| `healthy` | 表的健康度 |
+| `update_time` | 最近更新的时间 |
+| `last_analyze_time` | 上次收集统计信息的时间 |
+
+> 表的健康度：表示表统计信息的健康程度。当 `update_rows` 大于等于 `row_count` 时，健康度为 0；当 `update_rows` 小于 `row_count` 时，健康度为 `100 * (1 - update_rows / row_count)` 。
+
+列统计信息：
+
+| 信息 | 描述 |
+| :------------ | :------------------------- |
+| `row_count` | 列的总行数 |
Review Comment:
After table level stats supported, row_count in the col_stats is redandunt I think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "Kikyou1997 (via GitHub)" <gi...@apache.org>.
Kikyou1997 commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186655177
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
+| `healthy` | 表的健康度 |
+| `update_time` | 最近更新的时间 |
+| `last_analyze_time` | 上次收集统计信息的时间 |
+
+> 表的健康度：表示表统计信息的健康程度。当 `update_rows` 大于等于 `row_count` 时，健康度为 0；当 `update_rows` 小于 `row_count` 时，健康度为 `100 * (1 - update_rows / row_count)` 。
+
+列统计信息：
+
+| 信息 | 描述 |
+| :------------ | :------------------------- |
+| `row_count` | 列的总行数 |
+| `data_size` | 列的总⻓度（单位 byte） |
+| `avg_size_byte` | 列的平均⻓度（单位 bytes） |
+| `ndv` | 列 num distinct value |
+| `min` | 列最小值 |
+| `max` | 列最⼤值 |
+| `null_count` | 列 null 个数 |
+| `histogram` | 列直方图 |
+
+接下来将简单介绍其中出现的直方图等数据结构，以及详细介绍统计信息的收集和维护。
+
+## 直方图简介
+
+直方图（histogram）是一种用于描述数据分布情况的工具，它将数据根据大小分成若干个区间（桶），并使用简单的统计量来表示每个区间中数据的特征。是数据库中的一种重要的统计信息，可以描述列中的数据分布情况。直方图最典型的应用场景是通过估算查询谓词的选择率来帮助优化器选择最优的执行计划。
+
+在 Doris 中，会对每个表具体的列构建一个等高直方图（Equi-height Histogram）。直方图包括一系列桶， 其中每个桶的统计量包括桶的上下界、桶包含的元素数量、前面桶所有元素的数量、桶中不同值的个数。具体可以参考 SQL 函数 `histogram` 或者 `hist` 的使用说明。
+
+> 采用等高直方图的分桶方法，每个桶中的数值频次之和都应该接近于总行数的 `1/N`。但是，如果严格遵守等高原则进行分桶，会出现某些值落在桶的边界上的情况，导致同一个值出现在两个不同的桶中。这种情况会对选择率的估算造成干扰。因此，在实现中，Doris 对等高直方图的分桶方法进行了修改：如果将一个值加入到某个桶中导致该桶中的数据频次超过了总行数的 `1/N`，则根据哪种情况更接近 `1/N`，将该值放入该桶或下一个桶中。
+
+## 收集统计信息
+
+### 手动收集
+
+⽤户通过 `ANALYZE` 语句触发手动收集任务，根据提供的参数，收集指定的表或列的统计信息。
+
+列统计信息收集语法：
+
+```SQL
+ANALYZE [ SYNC ] TABLE table_name
Review Comment:
I think `ANALYZE [SYNC] TABLE tbl_name` could be removed. It's duplicated with `ANALYZE [SYNC] TABLE tbl_name` WITH SYNC`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1189499924
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
+| `healthy` | 表的健康度 |
Review Comment:
Same as above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] weizhengte commented on a diff in pull request #19323: [Doc](statistics) add statistics document
Posted by "weizhengte (via GitHub)" <gi...@apache.org>.
weizhengte commented on code in PR #19323:
URL: https://github.com/apache/doris/pull/19323#discussion_r1186655979
##########
docs/zh-CN/docs/query-acceleration/statistics.md:
##########
@@ -0,0 +1,888 @@
+---
+{
+"title": "统计信息",
+"language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 统计信息
+
+## 统计信息简介
+
+在 SQL 数据库中，查询优化器的质量对系统性能有重要影响。优化器需要依据统计信息来估算查询代价，尤其在等值查询场景中，精确估算基数是非常重要的，基数估算可以帮助优化器选择最优的查询计划，从而提高查询性能。
+
+在执行一个查询时，未经过充分优化的执行计划和经过优化的执行计划可能会导致执行时间上的巨大差异，这个差距可以高达数倍。因此，对于 SQL 查询优化器来说，收集和分析统计信息是非常重要的，只有通过这些统计信息，优化器才能够准确地评估不同执行计划的成本，并选择最佳的执行计划。
+
+Doris 查询优化器使用统计信息来确定查询最有效的执行计划。Doris 维护的统计信息包括表级别的统计信息和列级别的统计信息。
+
+表统计信息：
+
+| 信息 | 描述 |
+| :------------------ | :------------------------- |
+| `row_count` | 表的行数 |
+| `data_size` | 表的⼤⼩（单位 byte） |
+| `update_rows` | 收集统计信息后所更新的行数 |
+| `healthy` | 表的健康度 |
+| `update_time` | 最近更新的时间 |
+| `last_analyze_time` | 上次收集统计信息的时间 |
+
+> 表的健康度：表示表统计信息的健康程度。当 `update_rows` 大于等于 `row_count` 时，健康度为 0；当 `update_rows` 小于 `row_count` 时，健康度为 `100 * (1 - update_rows / row_count)` 。
+
+列统计信息：
+
+| 信息 | 描述 |
+| :------------ | :------------------------- |
+| `row_count` | 列的总行数 |
+| `data_size` | 列的总⻓度（单位 byte） |
+| `avg_size_byte` | 列的平均⻓度（单位 bytes） |
+| `ndv` | 列 num distinct value |
+| `min` | 列最小值 |
+| `max` | 列最⼤值 |
+| `null_count` | 列 null 个数 |
+| `histogram` | 列直方图 |
+
+接下来将简单介绍其中出现的直方图等数据结构，以及详细介绍统计信息的收集和维护。
+
+## 直方图简介
+
+直方图（histogram）是一种用于描述数据分布情况的工具，它将数据根据大小分成若干个区间（桶），并使用简单的统计量来表示每个区间中数据的特征。是数据库中的一种重要的统计信息，可以描述列中的数据分布情况。直方图最典型的应用场景是通过估算查询谓词的选择率来帮助优化器选择最优的执行计划。
+
+在 Doris 中，会对每个表具体的列构建一个等高直方图（Equi-height Histogram）。直方图包括一系列桶， 其中每个桶的统计量包括桶的上下界、桶包含的元素数量、前面桶所有元素的数量、桶中不同值的个数。具体可以参考 SQL 函数 `histogram` 或者 `hist` 的使用说明。
+
+> 采用等高直方图的分桶方法，每个桶中的数值频次之和都应该接近于总行数的 `1/N`。但是，如果严格遵守等高原则进行分桶，会出现某些值落在桶的边界上的情况，导致同一个值出现在两个不同的桶中。这种情况会对选择率的估算造成干扰。因此，在实现中，Doris 对等高直方图的分桶方法进行了修改：如果将一个值加入到某个桶中导致该桶中的数据频次超过了总行数的 `1/N`，则根据哪种情况更接近 `1/N`，将该值放入该桶或下一个桶中。
+
+## 收集统计信息
+
+### 手动收集
+
+⽤户通过 `ANALYZE` 语句触发手动收集任务，根据提供的参数，收集指定的表或列的统计信息。
+
+列统计信息收集语法：
+
+```SQL
+ANALYZE [ SYNC ] TABLE table_name
+ [ (column_name [, ...]) ]
+ [ [ WITH SYNC ] [ WITH INCREMENTAL ] [ WITH SAMPLE PERCENT | ROWS ] [ WITH PERIOD ] ]
+ [ PROPERTIES ("key" = "value", ...) ];
+```
+
+列直方图收集语法：
+
+```SQL
+ANALYZE [ SYNC ] TABLE table_name
Review Comment:
Same as above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org