You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by "handmadecode (via GitHub)" <gi...@apache.org> on 2023/04/14 15:02:12 UTC
[PR] DRILL-8421: Parquet microsecond columns (drill)
handmadecode opened a new pull request, #2793:
URL: https://github.com/apache/drill/pull/2793
# [DRILL-8421](https://issues.apache.org/jira/browse/DRILL-8421): Truncate parquet microsecond columns
## Description
The metadata min and max values of parquet microsecond columns are truncated to milliseconds, which is the time unit expected by the initial file pruning during filtering. Also, `TIME_MICROS` columns are read as 64-bit values before they are truncated to 32-bit milliseconds values. Previously they were read as 32-bit values, causing values > `Integer.MAX_VALUE` to be incorrect.
The second fix also addresses [DRILL-8423](https://issues.apache.org/jira/browse/DRILL-8423).
## Documentation
Bugfix only, no documentation changes
## Testing
Unit tests added in new test class `org.apache.drill.exec.store.parquet.TestMicrosecondColumns`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] DRILL-8421: Parquet microsecond columns (drill)
Posted by "cgivre (via GitHub)" <gi...@apache.org>.
cgivre commented on PR #2793:
URL: https://github.com/apache/drill/pull/2793#issuecomment-1512276311
@handmadecode https://github.com/apache/drill/pull/2794 has been merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] DRILL-8421: Parquet microsecond columns (drill)
Posted by "jnturton (via GitHub)" <gi...@apache.org>.
jnturton merged PR #2793:
URL: https://github.com/apache/drill/pull/2793
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] DRILL-8421: Parquet microsecond columns (drill)
Posted by "cgivre (via GitHub)" <gi...@apache.org>.
cgivre commented on PR #2793:
URL: https://github.com/apache/drill/pull/2793#issuecomment-1511473455
@handmadecode
Thanks for the contribution and welcome to Drill! Would you mind rebasing once [DRILL-8424] (https://github.com/apache/drill/pull/2794) is merged? There are some CI issues which will be fixed by that PR.
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] DRILL-8421: Parquet microsecond columns (drill)
Posted by "jnturton (via GitHub)" <gi...@apache.org>.
jnturton commented on PR #2793:
URL: https://github.com/apache/drill/pull/2793#issuecomment-1511592469
> Thanks for the contribution and welcome to Drill! Would you mind rebasing once https://github.com/apache/drill/pull/2794 is merged?
Heh, I just came here to type exactly this. I reviewed the code changes and they look great so really we just need the CI run after rebasing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] DRILL-8421: Parquet microsecond columns (drill)
Posted by "handmadecode (via GitHub)" <gi...@apache.org>.
handmadecode commented on PR #2793:
URL: https://github.com/apache/drill/pull/2793#issuecomment-1511492806
@cgivre thanks, happy to contribute. I will rebase when 8424 is merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] DRILL-8421: Parquet microsecond columns (drill)
Posted by "handmadecode (via GitHub)" <gi...@apache.org>.
handmadecode commented on PR #2793:
URL: https://github.com/apache/drill/pull/2793#issuecomment-1512661306
@jnturton Happy to help!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org