You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Paul Rogers (JIRA)" <ji...@apache.org> on 2016/12/23 20:34:58 UTC
[jira] [Comment Edited] (DRILL-5157) Multiple unit tests fail with
Parquet async Snappy error
[ https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15773588#comment-15773588 ]
Paul Rogers edited comment on DRILL-5157 at 12/23/16 8:34 PM:
--------------------------------------------------------------
Problem also occurs in {{TestConvertFunctions#testConvertFromConvertToInt}} when run on a Mac.
Fails when running this query (with correct parameters):
{code}
CREATE TABLE %s.%s as
SELECT convert_to(r_regionkey, 'INT') as ct
FROM cp.`tpch/region.parquet`
{code}
was (Author: paul-rogers):
Problem also occurs in {{TestConvertFunctions#testConvertFromConvertToInt}} when run on a Mac.
> Multiple unit tests fail with Parquet async Snappy error
> --------------------------------------------------------
>
> Key: DRILL-5157
> URL: https://issues.apache.org/jira/browse/DRILL-5157
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.9.0
> Reporter: Paul Rogers
> Assignee: Parth Chandra
>
> Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It fails with the following stack trace and the memory leak trace shown second.
> Strangely, this error appears only if the test is run as part of the overall suite. The error does not appear if the test is run individually in the debugger.
> Stack trace that seems to show that the code was trying to find a Snappy native library:
> {code}
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> at org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402)
> at org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169)
> at org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96)
> at org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219)
> at org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280)
> at org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250)
> at org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178)
> at org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130)
> at org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485)
> at org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479)
> at org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562)
> at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178)
> at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> …
> {code}
> Resulting memory leak if the test is allowed to complete:
> {code}
> java.lang.AssertionError: Query state is incorrect (expected: CANCELED, actual: FAILED) AND/OR
> Exception thrown: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: IllegalStateException: Allocator[op:3:0:15:ParquetRowGroupScan] closed with outstanding buffers allocated (2).
> Allocator(op:3:0:15:ParquetRowGroupScan) 1000000/393216/3162112/10000000000 (res/actual/peak/limit)
> child allocators: 0
> ledgers: 2
> ledger[3407] allocator: op:3:0:15:ParquetRowGroupScan), isOwning: true, size: 131072, references: 1, life: 73148192887288..0, allocatorManager: [3027, life: 73148192235794..0] holds 1 buffers.
> DrillBuf[4949], udle: [3028 0..131072]
> ledger[3471] allocator: op:3:0:15:ParquetRowGroupScan), isOwning: true, size: 262144, references: 1, life: 73148451288840..0, allocatorManager: [3091, life: 73148451257480..0] holds 1 buffers.
> DrillBuf[5017], udle: [3092 0..262144]
> reservations: 0
> Fragment 3:0
> [Error Id: 8502074b-f488-4a14-bf7d-a2a4480392cd on 172.30.1.67:31016]
> at org.apache.drill.exec.server.TestDrillbitResilience.assertStateCompleted(TestDrillbitResilience.java:861)
> at org.apache.drill.exec.server.TestDrillbitResilience.assertCancelledWithoutException(TestDrillbitResilience.java:876)
> at org.apache.drill.exec.server.TestDrillbitResilience.doMemoryLeaksWhenCancelled(TestDrillbitResilience.java:680)
> at org.apache.drill.exec.server.TestDrillbitResilience.memoryLeaksWhenCancelled(TestDrillbitResilience.java:647)
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)