You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/07/06 14:52:32 UTC

[GitHub] [arrow-datafusion] alamb opened a new pull request #690: Fix Date32 and Date64 parquet row group pruning

alamb opened a new pull request #690:
URL: https://github.com/apache/arrow-datafusion/pull/690


   # Which issue does this PR close?
   
   Fixes: https://github.com/apache/arrow-datafusion/issues/649 as found by @yordan-pavlov 
   
   Build on https://github.com/apache/arrow-datafusion/pull/657 and https://github.com/apache/arrow-datafusion/pull/689 so please review them first
   
    # Rationale for this change
   Parquet pruning is not working for `Date32` or `Date64`
   
   # What changes are included in this PR?
   Fixes parquet pruning on `Date32` and `Date64` columns by adding the appropriate expression evaluation machinery for Date32 and Date64. Creating arrays for testing I found super awkward and I filed https://github.com/apache/arrow-rs/issues/527 to try and track improving that situation 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] alamb commented on a change in pull request #690: Fix Date32 and Date64 parquet row group pruning

Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #690:
URL: https://github.com/apache/arrow-datafusion/pull/690#discussion_r665386724



##########
File path: datafusion/src/scalar.rs
##########
@@ -900,6 +900,7 @@ impl TryFrom<ScalarValue> for i64 {
     fn try_from(value: ScalarValue) -> Result<Self> {
         match value {
             ScalarValue::Int64(Some(inner_value))
+            | ScalarValue::Date64(Some(inner_value))

Review comment:
       This mirrors the clause above for `Date32` in `TryFrom<ScalarValue> for i32`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] alamb commented on pull request #690: Fix Date32 and Date64 parquet row group pruning

Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #690:
URL: https://github.com/apache/arrow-datafusion/pull/690#issuecomment-876366765


   @yordan-pavlov  this is ready for review. cc @Dandandan 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] alamb commented on pull request #690: Fix Date32 and Date64 parquet row group pruning

Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #690:
URL: https://github.com/apache/arrow-datafusion/pull/690#issuecomment-877612550


   Thank you @Dandandan 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] Dandandan merged pull request #690: Fix Date32 and Date64 parquet row group pruning

Posted by GitBox <gi...@apache.org>.
Dandandan merged pull request #690:
URL: https://github.com/apache/arrow-datafusion/pull/690


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org