You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Sahil Takiar (Jira)" <ji...@apache.org> on 2020/08/13 17:14:00 UTC
[jira] [Updated] (IMPALA-10083) Improve row count estimates when
stats are not available
[ https://issues.apache.org/jira/browse/IMPALA-10083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sahil Takiar updated IMPALA-10083:
----------------------------------
Description:
There are various improvements that we can make to estimate row count stats even if stats are not available for a table.
There are various factors to consider here:
* Handling for partitioned vs. non-partitioned tables
** Handling for partitioned tables can be a bit tricky if the table is in a mixed state - some partitions have row counts while other don't
* Interoperability with other systems such as Hive and Spark
* Users can run alter table statements to manually set the value of the row count
* Handling of corrupt stats vs. missing stats
** Corrupt stats are defined as stats value less than -1, or values of 0 when the underlying table has nonempty files
** Missing stats are stats that have just not been computed, and are marked as such with the value -1
The JIRA will be used to track the various improvements via sub-tasks.
was:
There are various improvements that we can make to estimate row count stats even if stats are not available for a table.
There are various factors to consider here:
* Handling for partitioned vs. non-partitioned tables
** Handling for partitioned tables can be a bit tricky if the table is in a mixed state - some partitions have row counts while other don't
* Interoperability with other systems such as Hive and Spark
* Users can run alter table statements to manually set the value of the row count
The JIRA will be used to track the various improvements via sub-tasks.
> Improve row count estimates when stats are not available
> --------------------------------------------------------
>
> Key: IMPALA-10083
> URL: https://issues.apache.org/jira/browse/IMPALA-10083
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Sahil Takiar
> Priority: Major
>
> There are various improvements that we can make to estimate row count stats even if stats are not available for a table.
> There are various factors to consider here:
> * Handling for partitioned vs. non-partitioned tables
> ** Handling for partitioned tables can be a bit tricky if the table is in a mixed state - some partitions have row counts while other don't
> * Interoperability with other systems such as Hive and Spark
> * Users can run alter table statements to manually set the value of the row count
> * Handling of corrupt stats vs. missing stats
> ** Corrupt stats are defined as stats value less than -1, or values of 0 when the underlying table has nonempty files
> ** Missing stats are stats that have just not been computed, and are marked as such with the value -1
> The JIRA will be used to track the various improvements via sub-tasks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org