[jira] [Commented] (DRILL-5157) Multiple Snappy versions on class path; causes unit test failures

["Paul Rogers (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15773994#comment-15773994 ] 

Paul Rogers commented on DRILL-5157:
------------------------------------

Turns out the Snappy errors are due to missing Maven dependency management for the Snappy jar. Drill includes three different versions, including our ancient version from 2012. (See above.) The result is two Snappy versions on the class path: a newer one and an older one. (Again, see above.) The newer versions work correctly, our older one fails on the Mac. (Known problem discussed on Slashdot.) For some reason, one is used when tests pass, the other when the error below occurs.

The solution is to move to the latest version of Snappy and add the proper Maven dependency management. Once that is done, life is good.

> Multiple Snappy versions on class path; causes unit test failures
> -----------------------------------------------------------------
>
>                 Key: DRILL-5157
>                 URL: https://issues.apache.org/jira/browse/DRILL-5157
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.9.0
>            Reporter: Paul Rogers
>            Assignee: Paul Rogers
>
> Run the {{TestDrillbitResilience.doMemoryLeaksWhenCancelled}} unit test. It fails with the following stack trace and the memory leak trace shown second.
> Strangely, this error appears only if the test is run as part of the overall suite. The error does not appear if the test is run individually in the debugger. This suggests that the problem described here is a side-effect of a problem created by an earlier test.
> Investigation (see notes below) reveal that this is a Maven dependency management problem.
> Stack trace that seems to show that the code was trying to find a Snappy native library:
> {code}
> java.lang.reflect.InvocationTargetException
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:317)
> 	at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:219)
> 	at org.xerial.snappy.Snappy.<clinit>(Snappy.java:44)
> 	at org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$DecompressionHelper.decompress(AsyncPageReader.java:402)
> 	at org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.decompress(AsyncPageReader.java:169)
> 	at org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.getDecompressedPageData(AsyncPageReader.java:96)
> 	at org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.nextInternal(AsyncPageReader.java:219)
> 	at org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:280)
> 	at org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:250)
> 	at org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:178)
> 	at org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.processPages(ColumnReader.java:130)
> 	at org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFieldsSerial(ParquetRecordReader.java:485)
> 	at org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:479)
> 	at org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:562)
> 	at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:178)
> 	at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> 	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> …
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)