You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/12/08 04:03:53 UTC

[GitHub] [arrow] XiaokunDing opened a new pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

XiaokunDing opened a new pull request #8866:
URL: https://github.com/apache/arrow/pull/8866


   Add the 'Statistics' interface in data source


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] XiaokunDing commented on a change in pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
XiaokunDing commented on a change in pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#discussion_r539061490



##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics
+#[derive(Clone)]
+pub struct Statistics {
+    /// The number of table rows
+    pub num_rows: i64,

Review comment:
       I think here use `usize` is good than `u64`, because  while we implement the interface we should check the max value  less than  MAX `usize` and the value  strongly dependent on compile target.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] github-actions[bot] commented on pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#issuecomment-740362942


   https://issues.apache.org/jira/browse/ARROW-10781


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] alamb closed pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
alamb closed pull request #8866:
URL: https://github.com/apache/arrow/pull/8866


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] seddonm1 commented on a change in pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
seddonm1 commented on a change in pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#discussion_r539028012



##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics
+#[derive(Clone)]
+pub struct Statistics {
+    /// The number of table rows
+    pub num_rows: i64,

Review comment:
       My understanding is that because the parquet reader produces` i64` which (assuming above 0) is less than `u64` whereas a `usize` _may_ be smaller depending on compile target.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] seddonm1 commented on pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
seddonm1 commented on pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#issuecomment-740414867


   this looks good 👍 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] XiaokunDing closed pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
XiaokunDing closed pull request #8866:
URL: https://github.com/apache/arrow/pull/8866


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] XiaokunDing commented on pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
XiaokunDing commented on pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#issuecomment-740507022


   @seddonm1 , Thanks for your help


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] alamb commented on a change in pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#discussion_r539236791



##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics
+#[derive(Clone)]
+pub struct Statistics {
+    /// The number of table rows
+    pub num_rows: i64,

Review comment:
       I think `usize` is good enough to begin with -- if we need to change it ot u64 for some reason in the future (e.g. to support > 4GB files on 32-bit architectures) that should be a straightforward change. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] seddonm1 commented on a change in pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
seddonm1 commented on a change in pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#discussion_r538750212



##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics
+#[derive(Clone)]
+pub struct Statistics {
+    /// The number of table rows
+    pub num_rows: i64,
+    /// total byte of the table rows
+    pub total_byte_size: i64,

Review comment:
       i64 was my suggestion to be consistent with https://github.com/apache/arrow/blob/master/rust/parquet/src/file/metadata.rs#L193
   
   Perhaps that should be change to be `u64` too.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] XiaokunDing commented on a change in pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
XiaokunDing commented on a change in pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#discussion_r539070656



##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics
+#[derive(Clone)]
+pub struct Statistics {
+    /// The number of table rows
+    pub num_rows: i64,
+    /// total byte of the table rows
+    pub total_byte_size: i64,

Review comment:
       > i64 was my suggestion to be consistent with https://github.com/apache/arrow/blob/master/rust/parquet/src/file/metadata.rs#L193
   > 
   > Perhaps that should be change to be `u64` too.
   
   maybe, we should be change it from https://github.com/sunchao/parquet-format-rs/blob/f1d56730e24c06c9cdc39bfdb8000131469f28db/src/parquet_format.rs#L3258




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] andygrove commented on a change in pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
andygrove commented on a change in pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#discussion_r538442723



##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics

Review comment:
       I agree. These should be considerd estimates IMO. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] XiaokunDing commented on a change in pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
XiaokunDing commented on a change in pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#discussion_r539020866



##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics
+#[derive(Clone)]
+pub struct Statistics {
+    /// The number of table rows
+    pub num_rows: i64,

Review comment:
       Thanks @alamb . Is it better to use `usize`?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] XiaokunDing commented on a change in pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
XiaokunDing commented on a change in pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#discussion_r539020866



##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics
+#[derive(Clone)]
+pub struct Statistics {
+    /// The number of table rows
+    pub num_rows: i64,

Review comment:
       Is it better to use `usize`




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] XiaokunDing commented on a change in pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
XiaokunDing commented on a change in pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#discussion_r539020866



##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics
+#[derive(Clone)]
+pub struct Statistics {
+    /// The number of table rows
+    pub num_rows: i64,

Review comment:
       Is it better to use `usize`?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] codecov-io removed a comment on pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
codecov-io removed a comment on pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#issuecomment-741556657


   # [Codecov](https://codecov.io/gh/apache/arrow/pull/8866?src=pr&el=h1) Report
   > Merging [#8866](https://codecov.io/gh/apache/arrow/pull/8866?src=pr&el=desc) (16a1dff) into [master](https://codecov.io/gh/apache/arrow/commit/68ef7d2900a70efeefe5e6148ab3cf0f1ae68c81?el=desc) (68ef7d2) will **decrease** coverage by `31.65%`.
   > The diff coverage is `0.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/arrow/pull/8866/graphs/tree.svg?width=650&height=150&src=pr&token=LpTCFbqVT1)](https://codecov.io/gh/apache/arrow/pull/8866?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #8866       +/-   ##
   ===========================================
   - Coverage   84.57%   52.92%   -31.66%     
   ===========================================
     Files         186      172       -14     
     Lines       46543    30710    -15833     
   ===========================================
   - Hits        39366    16253    -23113     
   - Misses       7177    14457     +7280     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8866?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [rust/datafusion/src/datasource/csv.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9kYXRhc291cmNlL2Nzdi5ycw==) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [rust/datafusion/src/datasource/memory.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9kYXRhc291cmNlL21lbW9yeS5ycw==) | `0.00% <0.00%> (-82.26%)` | :arrow_down: |
   | [rust/datafusion/src/datasource/parquet.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9kYXRhc291cmNlL3BhcnF1ZXQucnM=) | `0.00% <0.00%> (-97.00%)` | :arrow_down: |
   | [rust/datafusion/tests/dataframe.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3Rlc3RzL2RhdGFmcmFtZS5ycw==) | `0.00% <0.00%> (-77.97%)` | :arrow_down: |
   | [rust/datafusion/src/test/variable.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy90ZXN0L3ZhcmlhYmxlLnJz) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [rust/datafusion/src/optimizer/filter\_push\_down.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9vcHRpbWl6ZXIvZmlsdGVyX3B1c2hfZG93bi5ycw==) | `0.00% <0.00%> (-99.33%)` | :arrow_down: |
   | [rust/parquet/src/column/page.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9jb2x1bW4vcGFnZS5ycw==) | `0.00% <0.00%> (-98.69%)` | :arrow_down: |
   | [rust/parquet/src/record/api.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9yZWNvcmQvYXBpLnJz) | `0.00% <0.00%> (-98.11%)` | :arrow_down: |
   | [rust/parquet/src/arrow/arrow\_writer.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9hcnJvdy9hcnJvd193cml0ZXIucnM=) | `0.00% <0.00%> (-97.34%)` | :arrow_down: |
   | [rust/parquet/src/basic.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9iYXNpYy5ycw==) | `0.00% <0.00%> (-97.27%)` | :arrow_down: |
   | ... and [137 more](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8866?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8866?src=pr&el=footer). Last update [68ef7d2...16a1dff](https://codecov.io/gh/apache/arrow/pull/8866?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] alamb commented on pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#issuecomment-741717908


   I removed what appeared to be a erronious codecov report


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] XiaokunDing commented on a change in pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
XiaokunDing commented on a change in pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#discussion_r539032854



##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics

Review comment:
       Thanks @alamb I have fixed the comment.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] XiaokunDing commented on a change in pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
XiaokunDing commented on a change in pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#discussion_r539032854



##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics

Review comment:
       Thanks @alamb I have fixed the code.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] alamb commented on a change in pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#discussion_r538371128



##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics
+#[derive(Clone)]
+pub struct Statistics {
+    /// The number of table rows
+    pub num_rows: i64,

Review comment:
       ```suggestion
       /// The number of table rows
       pub num_rows: u64,
   ```

##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics

Review comment:
       I suggest clarifying in the comment if the statistics are meant to be a hint or accurate. Specifically, it would help to know if other parts of the system should rely on them being correct or if they are simply a hit.
   
   Maybe @andygrove  has some thoughts as he was the one who filed https://issues.apache.org/jira/browse/ARROW-10781

##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics
+#[derive(Clone)]
+pub struct Statistics {
+    /// The number of table rows
+    pub num_rows: i64,
+    /// total byte of the table rows
+    pub total_byte_size: i64,

Review comment:
       ```suggestion
       /// total size of the table, in bytes
       pub total_byte_size: u64,
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] XiaokunDing commented on a change in pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
XiaokunDing commented on a change in pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#discussion_r539061490



##########
File path: rust/datafusion/src/datasource/datasource.rs
##########
@@ -24,6 +24,15 @@ use crate::arrow::datatypes::SchemaRef;
 use crate::error::Result;
 use crate::physical_plan::ExecutionPlan;
 
+/// The table statistics
+#[derive(Clone)]
+pub struct Statistics {
+    /// The number of table rows
+    pub num_rows: i64,

Review comment:
       I think here use `usize` is good than u64, because  while we implement the interface we should check the max value  less than  MAX `usize` and the value  strongly dependent on compile target.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] codecov-io commented on pull request #8866: ARROW-10781:[Rust] [DataFusion] add the 'Statistics' interface in data source

Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #8866:
URL: https://github.com/apache/arrow/pull/8866#issuecomment-741556657


   # [Codecov](https://codecov.io/gh/apache/arrow/pull/8866?src=pr&el=h1) Report
   > Merging [#8866](https://codecov.io/gh/apache/arrow/pull/8866?src=pr&el=desc) (16a1dff) into [master](https://codecov.io/gh/apache/arrow/commit/68ef7d2900a70efeefe5e6148ab3cf0f1ae68c81?el=desc) (68ef7d2) will **decrease** coverage by `31.65%`.
   > The diff coverage is `0.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/arrow/pull/8866/graphs/tree.svg?width=650&height=150&src=pr&token=LpTCFbqVT1)](https://codecov.io/gh/apache/arrow/pull/8866?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #8866       +/-   ##
   ===========================================
   - Coverage   84.57%   52.92%   -31.66%     
   ===========================================
     Files         186      172       -14     
     Lines       46543    30710    -15833     
   ===========================================
   - Hits        39366    16253    -23113     
   - Misses       7177    14457     +7280     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8866?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [rust/datafusion/src/datasource/csv.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9kYXRhc291cmNlL2Nzdi5ycw==) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [rust/datafusion/src/datasource/memory.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9kYXRhc291cmNlL21lbW9yeS5ycw==) | `0.00% <0.00%> (-82.26%)` | :arrow_down: |
   | [rust/datafusion/src/datasource/parquet.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9kYXRhc291cmNlL3BhcnF1ZXQucnM=) | `0.00% <0.00%> (-97.00%)` | :arrow_down: |
   | [rust/datafusion/tests/dataframe.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3Rlc3RzL2RhdGFmcmFtZS5ycw==) | `0.00% <0.00%> (-77.97%)` | :arrow_down: |
   | [rust/datafusion/src/test/variable.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy90ZXN0L3ZhcmlhYmxlLnJz) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [rust/datafusion/src/optimizer/filter\_push\_down.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9vcHRpbWl6ZXIvZmlsdGVyX3B1c2hfZG93bi5ycw==) | `0.00% <0.00%> (-99.33%)` | :arrow_down: |
   | [rust/parquet/src/column/page.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9jb2x1bW4vcGFnZS5ycw==) | `0.00% <0.00%> (-98.69%)` | :arrow_down: |
   | [rust/parquet/src/record/api.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9yZWNvcmQvYXBpLnJz) | `0.00% <0.00%> (-98.11%)` | :arrow_down: |
   | [rust/parquet/src/arrow/arrow\_writer.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9hcnJvdy9hcnJvd193cml0ZXIucnM=) | `0.00% <0.00%> (-97.34%)` | :arrow_down: |
   | [rust/parquet/src/basic.rs](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9iYXNpYy5ycw==) | `0.00% <0.00%> (-97.27%)` | :arrow_down: |
   | ... and [137 more](https://codecov.io/gh/apache/arrow/pull/8866/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8866?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8866?src=pr&el=footer). Last update [68ef7d2...16a1dff](https://codecov.io/gh/apache/arrow/pull/8866?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org