You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Peter Edike <pe...@interswitchgroup.com> on 2018/08/07 16:10:15 UTC
Error Querying Parquet Files In Apache Drill 1.13
Hello everyone ,
Please I am having issues querying some parquet files generated using scoop1 in Apache Drill.
I checked the logs and I see the following exception everywhere
Aug 7, 2018 4:28:09 PM WARNING: org.apache.parquet.CorruptStatistics: Ignoring statistics because created_by could not be parsed (see PARQUET-251): parquet-mr (build 6aa21f8776625b5fa6b18059cfebe7549f2e00cb)
org.apache.parquet.VersionParser$VersionParseException: Could not parse created_by: parquet-mr (build 6aa21f8776625b5fa6b18059cfebe7549f2e00cb) using format: (.+) version ((.*) )?\(build ?(.*)\)
at org.apache.parquet.VersionParser.parse(VersionParser.java:112)
at org.apache.parquet.CorruptStatistics.shouldIgnoreStatistics(CorruptStatistics.java:66)
at org.apache.parquet.format.converter.ParquetMetadataConverter.fromParquetStatistics(ParquetMetadataConverter.java:264)
at org.apache.parquet.format.converter.ParquetMetadataConverter.fromParquetMetadata(ParquetMetadataConverter.java:568)
at org.apache.parquet.format.converter.ParquetMetadataConverter.readParquetMetadata(ParquetMetadataConverter.java:545)
at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:455)
at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:412)
at org.apache.drill.exec.store.parquet.Metadata$1.run(Metadata.java:435)
at org.apache.drill.exec.store.parquet.Metadata$1.run(Metadata.java:428)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1633)
at org.apache.drill.exec.store.parquet.Metadata.getParquetFileMetadata_v3(Metadata.java:428)
at org.apache.drill.exec.store.parquet.Metadata.access$100(Metadata.java:96)
at org.apache.drill.exec.store.parquet.Metadata$MetadataGatherer.runInner(Metadata.java:364)
at org.apache.drill.exec.store.parquet.Metadata$MetadataGatherer.runInner(Metadata.java:352)
at org.apache.drill.exec.store.TimedRunnable.run(TimedRunnable.java:56)
at org.apache.drill.exec.store.TimedRunnable$LatchedRunnable.run(TimedRunnable.java:98)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Please what can I do to mitigate against this
________________________________
Peter Edike
Senior Software Engineer
Research and Development, ENG
Engineering
[cid:image002.png@01D42E71.8D117260]
Office NO:
Mobile NO:
Email: peter.edike@interswitchgroup.com<ma...@interswitchgroup.com>
Interswitch Limited
1648C Oko-Awo Street, Victoria Island Lagos
Customer Contact Centre 0700-9065000
? http://www.interswitchgroup.com<http://www.interswitchgroup.com/>
[cid:image003.png@01D42E71.8D117260]<https://www.quickteller.com/loan-request>
This e-mail and all attachments transmitted with it remain the property of Interswitch Limited , the information contained herein are private confidential and intended solely for the use of the addressee. If you have received this e-mail in error, kindly notify the sender. If you are not the addressee, you should not disseminate, distribute or copy this e-mail. Kindly notify Interswitch immediately by email if you have received this email in error and delete this email and any attachment from your system Emails cannot be guaranteed to be secure or error free as the message and any attachments could be intercepted, corrupted, lost, delayed, incomplete or amended. the contents of this email or its attachments have been scanned for all viruses and all reasonable measures have been taken to ensure that no viruses are present. Interswitch Limited and its subsidiaries do not accept liability for damage caused by this email or any attachments.This message has been marked as CONFIDENTIAL on Tuesday, August 7, 2018 @ 5:10:31 PM
Re: Error Querying Parquet Files In Apache Drill 1.13
Posted by Nitin Pawar <ni...@gmail.com>.
There is an open issue for this already DRILL-5145
<https://issues.apache.org/jira/browse/DRILL-5145>
This should not fail your queries though, its a warning
On Tue, Aug 7, 2018 at 9:40 PM, Peter Edike <
peter.edike@interswitchgroup.com> wrote:
> Hello everyone ,
>
>
>
> Please I am having issues querying some parquet files generated using
> scoop1 in Apache Drill.
>
>
>
> I checked the logs and I see the following exception everywhere
>
>
>
> Aug 7, 2018 4:28:09 PM WARNING: org.apache.parquet.CorruptStatistics:
> Ignoring statistics because created_by could not be parsed (see
> PARQUET-251): parquet-mr (build 6aa21f8776625b5fa6b18059cfebe7549f2e00cb)
>
> org.apache.parquet.VersionParser$VersionParseException: Could not parse
> created_by: parquet-mr (build 6aa21f8776625b5fa6b18059cfebe7549f2e00cb)
> using format: (.+) version ((.*) )?\(build ?(.*)\)
>
> at org.apache.parquet.VersionParser.parse(VersionParser.java:112)
>
> at org.apache.parquet.CorruptStatistics.shouldIgnoreStatistics(
> CorruptStatistics.java:66)
>
> at org.apache.parquet.format.converter.ParquetMetadataConverter.
> fromParquetStatistics(ParquetMetadataConverter.java:264)
>
> at org.apache.parquet.format.converter.ParquetMetadataConverter.
> fromParquetMetadata(ParquetMetadataConverter.java:568)
>
> at org.apache.parquet.format.converter.ParquetMetadataConverter.
> readParquetMetadata(ParquetMetadataConverter.java:545)
>
> at org.apache.parquet.hadoop.ParquetFileReader.readFooter(
> ParquetFileReader.java:455)
>
> at org.apache.parquet.hadoop.ParquetFileReader.readFooter(
> ParquetFileReader.java:412)
>
> at org.apache.drill.exec.store.parquet.Metadata$1.run(
> Metadata.java:435)
>
> at org.apache.drill.exec.store.parquet.Metadata$1.run(
> Metadata.java:428)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:422)
>
> at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1633)
>
> at org.apache.drill.exec.store.parquet.Metadata.
> getParquetFileMetadata_v3(Metadata.java:428)
>
> at org.apache.drill.exec.store.parquet.Metadata.access$100(
> Metadata.java:96)
>
> at org.apache.drill.exec.store.parquet.Metadata$
> MetadataGatherer.runInner(Metadata.java:364)
>
> at org.apache.drill.exec.store.parquet.Metadata$
> MetadataGatherer.runInner(Metadata.java:352)
>
> at org.apache.drill.exec.store.TimedRunnable.run(
> TimedRunnable.java:56)
>
> at org.apache.drill.exec.store.TimedRunnable$LatchedRunnable.
> run(TimedRunnable.java:98)
>
> at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:511)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1149)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:624)
>
> at java.lang.Thread.run(Thread.java:748)
>
>
>
> Please what can I do to mitigate against this
>
>
> ------------------------------
>
> *Peter Edike*
> Senior Software Engineer
> Research and Development, ENG
> Engineering
>
> *Office NO: *
> *Mobile NO: *
> *Email:* peter.edike@interswitchgroup.com
> Interswitch Limited
> 1648C Oko-Awo Street, Victoria Island Lagos
> Customer Contact Centre 0700-9065000
> *ΓΌ* *http://www.interswitchgroup.com* <http://www.interswitchgroup.com/>
>
> <https://www.quickteller.com/loan-request>
>
> This e-mail and all attachments transmitted with it remain the property
> of Interswitch Limited , the information contained herein are private
> confidential and intended solely for the use of the addressee. If you have
> received this e-mail in error, kindly notify the sender. If you are not the
> addressee, you should not disseminate, distribute or copy this e-mail.
> Kindly notify Interswitch immediately by email if you have received this
> email in error and delete this email and any attachment from your system
> Emails cannot be guaranteed to be secure or error free as the message and
> any attachments could be intercepted, corrupted, lost, delayed, incomplete
> or amended. the contents of this email or its attachments have been scanned
> for all viruses and all reasonable measures have been taken to ensure that
> no viruses are present. Interswitch Limited and its subsidiaries do not
> accept liability for damage caused by this email or any attachments.This
> message has been marked as *CONFIDENTIAL *on *Tuesday, August 7, 2018* @ *5:10:31
> PM*
>
>
>
--
Nitin Pawar