You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Chris Westin (JIRA)" <ji...@apache.org> on 2015/04/13 20:50:12 UTC
[jira] [Commented] (DRILL-2767) Fragment error on TPCH Scale Factor
30 on a query that completed successfully previously
[ https://issues.apache.org/jira/browse/DRILL-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14492835#comment-14492835 ]
Chris Westin commented on DRILL-2767:
-------------------------------------
The first exception in drillbit1.log:
2015-04-11 01:43:51,155 [2ad783a9-2bfd-793b-e9ec-b1ef9d2f5afd:frag:1:3] DEBUG o.a.d.exec.physical.impl.ScanBatch - Failed to read the batch. Stopping...
org.apache.drill.common.exceptions.DrillRuntimeException: java.io.IOException: invalid distance code
at org.apache.drill.exec.store.hive.HiveRecordReader.next(HiveRecordReader.java:319) ~[drill-storage-hive-core-0.8.0.jar:0.8.0]
at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:165) ~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:68) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:99) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:58) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:163) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-0.8.0-rebuffed.jar:0.8.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
Caused by: java.io.IOException: invalid distance code
at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.inflateBytesDirect(Native Method) ~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.decompress(ZlibDecompressor.java:228) ~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:91) ~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:85) ~[hadoop-common-2.4.1.jar:na]
at java.io.InputStream.read(InputStream.java:101) ~[na:1.7.0_71]
at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:180) ~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216) ~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174) ~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:209) ~[hadoop-mapreduce-client-core-2.4.1.jar:na]
at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:47) ~[hadoop-mapreduce-client-core-2.4.1.jar:na]
at org.apache.drill.exec.store.hive.HiveRecordReader.next(HiveRecordReader.java:305) ~[drill-storage-hive-core-0.8.0.jar:0.8.0]
... 10 common frames omitted
2015-04-11 01:43:51,156 [2ad783a9-2bfd-793b-e9ec-b1ef9d2f5afd:frag:1:3] ERROR o.a.drill.exec.ops.FragmentContext - Fragment Context received failure.
First exception in drillbit2.log:
2015-04-11 01:43:51,130 [2ad783a9-2bfd-793b-e9ec-b1ef9d2f5afd:frag:1:8] DEBUG o.a.d.exec.physical.impl.ScanBatch - Failed to read the batch. Stopping...
org.apache.drill.common.exceptions.DrillRuntimeException: java.io.IOException: invalid code lengths set
at org.apache.drill.exec.store.hive.HiveRecordReader.next(HiveRecordReader.java:319) ~[drill-storage-hive-core-0.8.0.jar:0.8.0]
at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:165) ~[drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:68) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:99) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:58) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:163) [drill-java-exec-0.8.0-rebuffed.jar:0.8.0]
at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-0.8.0-rebuffed.jar:0.8.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
Caused by: java.io.IOException: invalid code lengths set
at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.inflateBytesDirect(Native Method) ~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.decompress(ZlibDecompressor.java:228) ~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:91) ~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:85) ~[hadoop-common-2.4.1.jar:na]
at java.io.InputStream.read(InputStream.java:101) ~[na:1.7.0_71]
at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:180) ~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216) ~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174) ~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:209) ~[hadoop-mapreduce-client-core-2.4.1.jar:na]
at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:47) ~[hadoop-mapreduce-client-core-2.4.1.jar:na]
at org.apache.drill.exec.store.hive.HiveRecordReader.next(HiveRecordReader.java:305) ~[drill-storage-hive-core-0.8.0.jar:0.8.0]
... 10 common frames omitted
2015-04-11 01:43:51,131 [2ad783a9-2bfd-793b-e9ec-b1ef9d2f5afd:frag:1:8] ERROR o.a.drill.exec.ops.FragmentContext - Fragment Context received failure.
org.apache.drill.common.exceptions.DrillRuntimeException: java.io.IOException: invalid code lengths set
Is there a problem reading compressed stuff from hive?
> Fragment error on TPCH Scale Factor 30 on a query that completed successfully previously
> ----------------------------------------------------------------------------------------
>
> Key: DRILL-2767
> URL: https://issues.apache.org/jira/browse/DRILL-2767
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Hive
> Environment: AWS EMR cluster of three m1.xlarge nodes
> Reporter: Alexander Zarei
> Attachments: drillbitcore1.log, drillbitcore1.out, drillbitcore2.log, drillbitcore2.out, drillbitmaster.out
>
>
> The following sequence led to the error:
> Executed the query
> bq. SELECT * FROM `realhive`.`tpch_text_30`.`lineitem`
> and it took about 43 minutes to execute successfully.
> After ward I ran the query
> bq. SELECT * FROM `realhive`.`tpch_text_2`.`lineitem`
> for 6 times to find an optimization value for the ODBC driver.
> Afterward, I submitted the first query again
> bq. SELECT * FROM `realhive`.`tpch_text_30`.`lineitem`
>
> and the Drill Cluster returned a fragment error.
> Log files with debug level for the Drillbits on the master node as well as the core nodes of the cluster are attached.
> Also the connection through the ODBC driver on Linux 32 bit was "Direct" to the drillbit on the master node of the Hadoop cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)