You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Tanujit Ghosh <ta...@gmail.com> on 2013/08/27 06:06:34 UTC
Error with ParquetRecordReaderTest
Hi,
I'm facing this issue when i try to run the test in
org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
[tanujit@legolas java-exec]$ mvn
-Dtest=ParquetRecordReaderTest#testMultipleRowGroupsAndReads test
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for org.apache.drill.exec:java-exec:jar:1.0-SNAPSHOT
[WARNING] Detected profiles.xml alongside
org.apache.drill.exec:java-exec:1.0-SNAPSHOT, this file is no longer
supported and was ignored, please use the settings.xml instead
[WARNING]
[WARNING] It is highly recommended to fix these problems because they
threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support
building such malformed projects.
[WARNING]
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building java-exec 1.0-SNAPSHOT
[INFO]
------------------------------------------------------------------------
Downloading:
http://repository.apache.org/snapshots/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
Downloading:
http://conjars.org/repo/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
Downloading:
https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
Downloading:
http://repository.apache.org/snapshots/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
Downloading:
https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
Downloading:
http://conjars.org/repo/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
Downloading:
http://repository.apache.org/snapshots/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
Downloading:
http://conjars.org/repo/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
Downloading:
https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
[INFO]
[INFO] --- maven-resources-plugin:2.6:copy-resources (copy-resources) @
java-exec ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 370 resources
[INFO]
[INFO] --- maven-enforcer-plugin:1.2:enforce (no_commons_logging) @
java-exec ---
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (generate-sources) @ java-exec ---
[WARNING] Parameter tasks is deprecated, use target instead
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO] Registering compile source root
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/generated-sources
[INFO]
[INFO] --- fmpp-maven-plugin:1.0:generate (generate-sources) @ java-exec ---
Downloading:
https://oss.sonatype.org/content/repositories/snapshots/org/freemarker/freemarker/maven-metadata.xml
Downloading:
https://oss.sonatype.org/content/repositories/snapshots/oro/oro/maven-metadata.xml
Downloading:
https://oss.sonatype.org/content/repositories/snapshots/org/beanshell/bsh/maven-metadata.xml
Downloading:
https://oss.sonatype.org/content/repositories/snapshots/xml-resolver/xml-resolver/maven-metadata.xml
- Executing: ValueHolders.java
log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
- Executing: VariableLengthVectors.java
- Executing: FixedValueVectors.java
- Executing: TypeHelper.java
- Executing: NullableValueVectors.java
- Executing: RepeatedValueVectors.java
[INFO] Done
[INFO]
[INFO] --- maven-remote-resources-plugin:1.1:process (default) @ java-exec
---
[INFO] Setting property: classpath.resource.loader.class =>
'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'.
[INFO] Setting property: velocimacro.messages.on => 'false'.
[INFO] Setting property: resource.loader => 'classpath'.
[INFO] Setting property: resource.manager.logwhenfound => 'false'.
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
java-exec ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @ java-exec
---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 460 source files to
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/classes
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetStorageEngine.java:[20,21]
com.sun.corba.se.impl.interceptors.CodecFactoryImpl is internal proprietary
API and may be removed in a future release
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/bit/BitConnection.java:
Some input files use or override a deprecated API.
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/bit/BitConnection.java:
Recompile with -Xlint:deprecation for details.
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java:
Some input files use unchecked or unsafe operations.
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java:
Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources)
@ java-exec ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 37 resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.0:testCompile (default-testCompile) @
java-exec ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 127 source files to
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/test-classes
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[25,16]
sun.misc.Unsafe is internal proprietary API and may be removed in a future
release
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[53,17]
sun.misc.Unsafe is internal proprietary API and may be removed in a future
release
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[94,12]
sun.misc.Unsafe is internal proprietary API and may be removed in a future
release
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[95,14]
sun.misc.Unsafe is internal proprietary API and may be removed in a future
release
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[98,26]
sun.misc.Unsafe is internal proprietary API and may be removed in a future
release
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[100,27]
sun.misc.Unsafe is internal proprietary API and may be removed in a future
release
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/TestParquetPhysicalPlan.java:
Some input files use or override a deprecated API.
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/TestParquetPhysicalPlan.java:
Recompile with -Xlint:deprecation for details.
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java:
Some input files use unchecked or unsafe operations.
[WARNING]
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java:
Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-surefire-plugin:2.15:test (default-test) @ java-exec ---
[INFO] Surefire report directory:
/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
09:23:52.569 [main] DEBUG org.apache.hadoop.security.Groups - Creating new
Groups object
09:23:52.662 [main] DEBUG org.apache.hadoop.security.Groups - Group mapping
impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping;
cacheTimeout=300000
09:23:52.841 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop login
09:23:52.842 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop
login commit
09:23:52.863 [main] DEBUG o.a.h.security.UserGroupInformation - using local
user:UnixPrincipal: tanujit
09:23:52.867 [main] DEBUG o.a.h.security.UserGroupInformation - UGI
loginUser:tanujit
09:23:52.921 [main] DEBUG org.apache.hadoop.fs.FileSystem - Creating
filesystem for file:///
09:23:53.011 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - Trying to load
the custom-built native-hadoop library...
09:23:53.033 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - Failed to load
native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in
java.library.path
09:23:53.034 [main] DEBUG o.a.hadoop.util.NativeCodeLoader -
java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
09:23:53.034 [main] WARN o.a.hadoop.util.NativeCodeLoader - Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
09:23:55.443 [main] DEBUG org.reflections.Reflections - going to scan these
urls:
jar:file:/home/tanujit/.m2/repository/org/apache/drill/common/1.0-SNAPSHOT/common-1.0-SNAPSHOT.jar!/
file:/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/test-classes/
file:/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/classes/
jar:file:/home/tanujit/.m2/repository/org/apache/drill/common/1.0-SNAPSHOT/common-1.0-SNAPSHOT-tests.jar!/
09:23:56.000 [main] INFO org.reflections.Reflections - Reflections took
556 ms to scan 4 urls, producing 669 keys and 1634 values
09:23:56.028 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as
the default logging framework
09:23:56.037 [main] DEBUG i.n.c.MultithreadEventLoopGroup -
-Dio.netty.eventLoopThreads: 8
09:23:56.074 [main] DEBUG i.n.util.internal.PlatformDependent - UID: 1000
09:23:56.075 [main] DEBUG i.n.util.internal.PlatformDependent - Java
version: 7
09:23:56.076 [main] DEBUG i.n.util.internal.PlatformDependent -
-Dio.netty.noUnsafe: false
09:23:56.079 [main] DEBUG i.n.util.internal.PlatformDependent0 -
java.nio.ByteBuffer.cleaner: available
09:23:56.079 [main] DEBUG i.n.util.internal.PlatformDependent0 -
java.nio.Buffer.address: available
09:23:56.080 [main] DEBUG i.n.util.internal.PlatformDependent0 -
sun.misc.Unsafe.theUnsafe: available
09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent0 -
sun.misc.Unsafe.copyMemory: available
09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent0 -
java.nio.Bits.unaligned: true
09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent -
sun.misc.Unsafe: available
09:23:56.082 [main] DEBUG i.n.util.internal.PlatformDependent -
-Dio.netty.noJavassist: false
09:23:56.145 [main] DEBUG i.n.util.internal.PlatformDependent - Javassist:
available
09:23:56.145 [main] DEBUG i.n.util.internal.PlatformDependent -
-Dio.netty.noPreferDirect: false
09:23:56.164 [main] DEBUG io.netty.channel.nio.NioEventLoop -
-Dio.netty.noKeySetOptimization: false
09:23:56.164 [main] DEBUG io.netty.channel.nio.NioEventLoop -
-Dio.netty.selectorAutoRebuildThreshold: 512
09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
-Dio.netty.allocator.numHeapArenas: 4
09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
-Dio.netty.allocator.numDirectArenas: 4
09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
-Dio.netty.allocator.pageSize: 8192
09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
-Dio.netty.allocator.maxOrder: 11
09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
-Dio.netty.allocator.chunkSize: 16777216
running bit
09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback interface: lo
09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback address:
/0:0:0:0:0:0:0:1%1 (primary)
09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback address: /
127.0.0.1
09:23:56.277 [main] DEBUG io.netty.util.NetUtil -
/proc/sys/net/core/somaxconn: 128
09:23:56.288 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
0x036be036] REGISTERED
09:23:56.290 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
0x036be036] BIND(0.0.0.0/0.0.0.0:31010)
09:23:56.294 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
0x036be036, /0:0:0:0:0:0:0:0:31010] ACTIVE
09:23:56.310 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
0xb8299b35] REGISTERED
09:23:56.311 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
0xb8299b35] BIND(0.0.0.0/0.0.0.0:31011)
09:23:56.311 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
0xb8299b35, /0:0:0:0:0:0:0:0:31011] ACTIVE
running client
connecting - legolas -- 31010
09:23:56.545 [Client-1] DEBUG
i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
io.netty.util.internal.__matchers__.io.netty.buffer.ByteBufMatcher
09:23:56.549 [Client-1] DEBUG
i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
io.netty.util.internal.__matchers__.org.apache.drill.exec.rpc.OutboundRpcMessageMatcher
09:23:56.550 [Client-1] DEBUG
i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
io.netty.util.internal.__matchers__.org.apache.drill.exec.rpc.InboundRpcMessageMatcher
09:23:56.554 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
0x036be036, /0:0:0:0:0:0:0:0:31010] RECEIVED: [id: 0xf046bccd, /
192.168.0.103:32952 => /192.168.0.103:31010]
09:23:56.568 [Client-1] DEBUG io.netty.util.ResourceLeakDetector -
-Dio.netty.noResourceLeakDetection: false
running the query
Screen [endpoint=address: "legolas"
user_port: 31010
bit_port: 31011
, getChild()=org.apache.drill.exec.store.parquet.ParquetGroupScan@59c81460]
09:24:00.420 [WorkManager-2] WARN io.netty.util.ResourceLeakDetector -
LEAK: ByteBuf was GC'd before being released correctly. The following
stack trace shows where the leaked object was created, rather than where
you failed to release it.
io.netty.util.ResourceLeakException:
io.netty.buffer.PooledUnsafeDirectByteBufL@659974ba
at
io.netty.util.ResourceLeakDetector$DefaultResourceLeak.<init>(ResourceLeakDetector.java:174)
~[netty-common-4.0.7.Final.jar:na]
at io.netty.util.ResourceLeakDetector.open(ResourceLeakDetector.java:116)
~[netty-common-4.0.7.Final.jar:na]
at io.netty.buffer.PooledByteBufL.<init>(PooledByteBufL.java:45)
~[netty-bufferl-4.0.7.Final.jar:na]
at
io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:53)
~[netty-bufferl-4.0.7.Final.jar:na]
at
io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:32)
~[netty-bufferl-4.0.7.Final.jar:na]
at
io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:39)
~[netty-bufferl-4.0.7.Final.jar:na]
at
io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:36)
~[netty-bufferl-4.0.7.Final.jar:na]
at io.netty.util.Recycler.get(Recycler.java:40)
~[netty-common-4.0.7.Final.jar:na]
at
io.netty.buffer.PooledUnsafeDirectByteBufL.newInstance(PooledUnsafeDirectByteBufL.java:44)
~[netty-bufferl-4.0.7.Final.jar:na]
at io.netty.buffer.PoolArenaL$DirectArena.newByteBuf(PoolArenaL.java:397)
~[netty-bufferl-4.0.7.Final.jar:na]
at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:93)
~[netty-bufferl-4.0.7.Final.jar:na]
at
io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:235)
~[netty-bufferl-4.0.7.Final.jar:na]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132)
~[netty-buffer-4.0.7.Final.jar:na]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123)
~[netty-buffer-4.0.7.Final.jar:na]
at
org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35)
~[classes/:na]
at org.apache.drill.exec.vector.BitVector.allocateNew(BitVector.java:54)
~[classes/:na]
at
org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:12)
~[classes/:na]
at
org.apache.drill.exec.store.VectorHolder.allocateNew(VectorHolder.java:87)
~[classes/:na]
at org.apache.drill.exec.store.VectorHolder.reset(VectorHolder.java:74)
~[classes/:na]
at
org.apache.drill.exec.store.parquet.ParquetRecordReader.resetBatch(ParquetRecordReader.java:276)
~[classes/:na]
at
org.apache.drill.exec.store.parquet.ParquetRecordReader.next(ParquetRecordReader.java:318)
~[classes/:na]
at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:99)
~[classes/:na]
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:77)
~[classes/:na]
at org.apache.drill.exec.work.FragmentRunner.run(FragmentRunner.java:79)
~[classes/:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_25]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.7.0_25]
at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
Took 10003 ms to run query
09:24:07.614 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
0xb8299b35, /0:0:0:0:0:0:0:0:31011] UNREGISTERED
09:24:07.614 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
0x036be036, /0:0:0:0:0:0:0:0:31010] UNREGISTERED
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.329 sec
- in org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
09:24:07.621 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Starting
clear of FileSystem cache with 1 elements.
09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Removing
filesystem for file:///
09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Removing
filesystem for file:///
09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Done
clearing cache
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 55.501s
[INFO] Finished at: Tue Aug 27 09:24:07 IST 2013
[INFO] Final Memory: 36M/429M
[INFO]
------------------------------------------------------------------------
--
Regards,
Tanujit
Re: Error with ParquetRecordReaderTest
Posted by ta...@gmail.com.
Is the excetion about a leak supposed to happen ?
Sent from my android device.
-----Original Message-----
From: Steven Phillips <sp...@maprtech.com>
To: drill-dev@incubator.apache.org
Sent: Tue, 27 Aug 2013 10:48
Subject: Re: Error with ParquetRecordReaderTest
Maybe I'm just not looking closely enough, but I don't see the problem. It
looks like the test passed without issue.
On Mon, Aug 26, 2013 at 9:06 PM, Tanujit Ghosh <ta...@gmail.com>wrote:
> Hi,
>
> I'm facing this issue when i try to run the test in
>
> org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
>
> [tanujit@legolas java-exec]$ mvn
> -Dtest=ParquetRecordReaderTest#testMultipleRowGroupsAndReads test
> [INFO] Scanning for projects...
> [WARNING]
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.drill.exec:java-exec:jar:1.0-SNAPSHOT
> [WARNING] Detected profiles.xml alongside
> org.apache.drill.exec:java-exec:1.0-SNAPSHOT, this file is no longer
> supported and was ignored, please use the settings.xml instead
> [WARNING]
> [WARNING] It is highly recommended to fix these problems because they
> threaten the stability of your build.
> [WARNING]
> [WARNING] For this reason, future Maven versions might no longer support
> building such malformed projects.
> [WARNING]
> [INFO]
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Building java-exec 1.0-SNAPSHOT
> [INFO]
> ------------------------------------------------------------------------
> Downloading:
>
> http://repository.apache.org/snapshots/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> http://conjars.org/repo/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> http://repository.apache.org/snapshots/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> http://conjars.org/repo/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> http://repository.apache.org/snapshots/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> http://conjars.org/repo/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
> [INFO]
> [INFO] --- maven-resources-plugin:2.6:copy-resources (copy-resources) @
> java-exec ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 370 resources
> [INFO]
> [INFO] --- maven-enforcer-plugin:1.2:enforce (no_commons_logging) @
> java-exec ---
> [INFO]
> [INFO] --- maven-antrun-plugin:1.6:run (generate-sources) @ java-exec ---
> [WARNING] Parameter tasks is deprecated, use target instead
> [INFO] Executing tasks
>
> main:
> [INFO] Executed tasks
> [INFO] Registering compile source root
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/generated-sources
> [INFO]
> [INFO] --- fmpp-maven-plugin:1.0:generate (generate-sources) @ java-exec
> ---
> Downloading:
>
> https://oss.sonatype.org/content/repositories/snapshots/org/freemarker/freemarker/maven-metadata.xml
> Downloading:
>
> https://oss.sonatype.org/content/repositories/snapshots/oro/oro/maven-metadata.xml
> Downloading:
>
> https://oss.sonatype.org/content/repositories/snapshots/org/beanshell/bsh/maven-metadata.xml
> Downloading:
>
> https://oss.sonatype.org/content/repositories/snapshots/xml-resolver/xml-resolver/maven-metadata.xml
> - Executing: ValueHolders.java
> log4j:WARN No appenders could be found for logger (freemarker.cache).
> log4j:WARN Please initialize the log4j system properly.
> - Executing: VariableLengthVectors.java
> - Executing: FixedValueVectors.java
> - Executing: TypeHelper.java
> - Executing: NullableValueVectors.java
> - Executing: RepeatedValueVectors.java
> [INFO] Done
> [INFO]
> [INFO] --- maven-remote-resources-plugin:1.1:process (default) @ java-exec
> ---
> [INFO] Setting property: classpath.resource.loader.class =>
> 'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'.
> [INFO] Setting property: velocimacro.messages.on => 'false'.
> [INFO] Setting property: resource.loader => 'classpath'.
> [INFO] Setting property: resource.manager.logwhenfound => 'false'.
> [INFO]
> [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
> java-exec ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 1 resource
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @ java-exec
> ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 460 source files to
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/classes
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetStorageEngine.java:[20,21]
> com.sun.corba.se.impl.interceptors.CodecFactoryImpl is internal proprietary
> API and may be removed in a future release
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/bit/BitConnection.java:
> Some input files use or override a deprecated API.
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/bit/BitConnection.java:
> Recompile with -Xlint:deprecation for details.
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java:
> Some input files use unchecked or unsafe operations.
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java:
> Recompile with -Xlint:unchecked for details.
> [INFO]
> [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources)
> @ java-exec ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 37 resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:3.0:testCompile (default-testCompile) @
> java-exec ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 127 source files to
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/test-classes
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[25,16]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[53,17]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[94,12]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[95,14]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[98,26]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[100,27]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/TestParquetPhysicalPlan.java:
> Some input files use or override a deprecated API.
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/TestParquetPhysicalPlan.java:
> Recompile with -Xlint:deprecation for details.
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java:
> Some input files use unchecked or unsafe operations.
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java:
> Recompile with -Xlint:unchecked for details.
> [INFO]
> [INFO] --- maven-surefire-plugin:2.15:test (default-test) @ java-exec ---
> [INFO] Surefire report directory:
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/surefire-reports
>
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
> 09:23:52.569 [main] DEBUG org.apache.hadoop.security.Groups - Creating new
> Groups object
> 09:23:52.662 [main] DEBUG org.apache.hadoop.security.Groups - Group mapping
> impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping;
> cacheTimeout=300000
> 09:23:52.841 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop
> login
> 09:23:52.842 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop
> login commit
> 09:23:52.863 [main] DEBUG o.a.h.security.UserGroupInformation - using local
> user:UnixPrincipal: tanujit
> 09:23:52.867 [main] DEBUG o.a.h.security.UserGroupInformation - UGI
> loginUser:tanujit
> 09:23:52.921 [main] DEBUG org.apache.hadoop.fs.FileSystem - Creating
> filesystem for file:///
> 09:23:53.011 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - Trying to load
> the custom-built native-hadoop library...
> 09:23:53.033 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - Failed to load
> native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in
> java.library.path
> 09:23:53.034 [main] DEBUG o.a.hadoop.util.NativeCodeLoader -
>
> java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 09:23:53.034 [main] WARN o.a.hadoop.util.NativeCodeLoader - Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> 09:23:55.443 [main] DEBUG org.reflections.Reflections - going to scan these
> urls:
>
> jar:file:/home/tanujit/.m2/repository/org/apache/drill/common/1.0-SNAPSHOT/common-1.0-SNAPSHOT.jar!/
>
> file:/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/test-classes/
>
> file:/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/classes/
>
> jar:file:/home/tanujit/.m2/repository/org/apache/drill/common/1.0-SNAPSHOT/common-1.0-SNAPSHOT-tests.jar!/
>
> 09:23:56.000 [main] INFO org.reflections.Reflections - Reflections took
> 556 ms to scan 4 urls, producing 669 keys and 1634 values
> 09:23:56.028 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as
> the default logging framework
> 09:23:56.037 [main] DEBUG i.n.c.MultithreadEventLoopGroup -
> -Dio.netty.eventLoopThreads: 8
> 09:23:56.074 [main] DEBUG i.n.util.internal.PlatformDependent - UID: 1000
> 09:23:56.075 [main] DEBUG i.n.util.internal.PlatformDependent - Java
> version: 7
> 09:23:56.076 [main] DEBUG i.n.util.internal.PlatformDependent -
> -Dio.netty.noUnsafe: false
> 09:23:56.079 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> java.nio.ByteBuffer.cleaner: available
> 09:23:56.079 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> java.nio.Buffer.address: available
> 09:23:56.080 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> sun.misc.Unsafe.theUnsafe: available
> 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> sun.misc.Unsafe.copyMemory: available
> 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> java.nio.Bits.unaligned: true
> 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent -
> sun.misc.Unsafe: available
> 09:23:56.082 [main] DEBUG i.n.util.internal.PlatformDependent -
> -Dio.netty.noJavassist: false
> 09:23:56.145 [main] DEBUG i.n.util.internal.PlatformDependent - Javassist:
> available
> 09:23:56.145 [main] DEBUG i.n.util.internal.PlatformDependent -
> -Dio.netty.noPreferDirect: false
> 09:23:56.164 [main] DEBUG io.netty.channel.nio.NioEventLoop -
> -Dio.netty.noKeySetOptimization: false
> 09:23:56.164 [main] DEBUG io.netty.channel.nio.NioEventLoop -
> -Dio.netty.selectorAutoRebuildThreshold: 512
> 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> -Dio.netty.allocator.numHeapArenas: 4
> 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> -Dio.netty.allocator.numDirectArenas: 4
> 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> -Dio.netty.allocator.pageSize: 8192
> 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> -Dio.netty.allocator.maxOrder: 11
> 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> -Dio.netty.allocator.chunkSize: 16777216
> running bit
> 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback interface: lo
> 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback address:
> /0:0:0:0:0:0:0:1%1 (primary)
> 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback address: /
> 127.0.0.1
> 09:23:56.277 [main] DEBUG io.netty.util.NetUtil -
> /proc/sys/net/core/somaxconn: 128
> 09:23:56.288 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0x036be036] REGISTERED
> 09:23:56.290 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0x036be036] BIND(0.0.0.0/0.0.0.0:31010)
> 09:23:56.294 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0x036be036, /0:0:0:0:0:0:0:0:31010] ACTIVE
> 09:23:56.310 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0xb8299b35] REGISTERED
> 09:23:56.311 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0xb8299b35] BIND(0.0.0.0/0.0.0.0:31011)
> 09:23:56.311 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0xb8299b35, /0:0:0:0:0:0:0:0:31011] ACTIVE
> running client
> connecting - legolas -- 31010
> 09:23:56.545 [Client-1] DEBUG
> i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
> io.netty.util.internal.__matchers__.io.netty.buffer.ByteBufMatcher
> 09:23:56.549 [Client-1] DEBUG
> i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
>
> io.netty.util.internal.__matchers__.org.apache.drill.exec.rpc.OutboundRpcMessageMatcher
> 09:23:56.550 [Client-1] DEBUG
> i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
>
> io.netty.util.internal.__matchers__.org.apache.drill.exec.rpc.InboundRpcMessageMatcher
> 09:23:56.554 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0x036be036, /0:0:0:0:0:0:0:0:31010] RECEIVED: [id: 0xf046bccd, /
> 192.168.0.103:32952 => /192.168.0.103:31010]
> 09:23:56.568 [Client-1] DEBUG io.netty.util.ResourceLeakDetector -
> -Dio.netty.noResourceLeakDetection: false
> running the query
> Screen [endpoint=address: "legolas"
> user_port: 31010
> bit_port: 31011
> , getChild()=org.apache.drill.exec.store.parquet.ParquetGroupScan@59c81460
> ]
> 09:24:00.420 [WorkManager-2] WARN io.netty.util.ResourceLeakDetector -
> LEAK: ByteBuf was GC'd before being released correctly. The following
> stack trace shows where the leaked object was created, rather than where
> you failed to release it.
> io.netty.util.ResourceLeakException:
> io.netty.buffer.PooledUnsafeDirectByteBufL@659974ba
> at
>
> io.netty.util.ResourceLeakDetector$DefaultResourceLeak.<init>(ResourceLeakDetector.java:174)
> ~[netty-common-4.0.7.Final.jar:na]
> at io.netty.util.ResourceLeakDetector.open(ResourceLeakDetector.java:116)
> ~[netty-common-4.0.7.Final.jar:na]
> at io.netty.buffer.PooledByteBufL.<init>(PooledByteBufL.java:45)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:53)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:32)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:39)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:36)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at io.netty.util.Recycler.get(Recycler.java:40)
> ~[netty-common-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.PooledUnsafeDirectByteBufL.newInstance(PooledUnsafeDirectByteBufL.java:44)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at io.netty.buffer.PoolArenaL$DirectArena.newByteBuf(PoolArenaL.java:397)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:93)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:235)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132)
> ~[netty-buffer-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123)
> ~[netty-buffer-4.0.7.Final.jar:na]
> at
>
> org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35)
> ~[classes/:na]
> at org.apache.drill.exec.vector.BitVector.allocateNew(BitVector.java:54)
> ~[classes/:na]
> at
>
> org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:12)
> ~[classes/:na]
> at
> org.apache.drill.exec.store.VectorHolder.allocateNew(VectorHolder.java:87)
> ~[classes/:na]
> at org.apache.drill.exec.store.VectorHolder.reset(VectorHolder.java:74)
> ~[classes/:na]
> at
>
> org.apache.drill.exec.store.parquet.ParquetRecordReader.resetBatch(ParquetRecordReader.java:276)
> ~[classes/:na]
> at
>
> org.apache.drill.exec.store.parquet.ParquetRecordReader.next(ParquetRecordReader.java:318)
> ~[classes/:na]
> at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:99)
> ~[classes/:na]
> at
>
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:77)
> ~[classes/:na]
> at org.apache.drill.exec.work.FragmentRunner.run(FragmentRunner.java:79)
> ~[classes/:na]
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_25]
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ~[na:1.7.0_25]
> at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
> Took 10003 ms to run query
> 09:24:07.614 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0xb8299b35, /0:0:0:0:0:0:0:0:31011] UNREGISTERED
> 09:24:07.614 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0x036be036, /0:0:0:0:0:0:0:0:31010] UNREGISTERED
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.329 sec
> - in org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
> 09:24:07.621 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Starting
> clear of FileSystem cache with 1 elements.
> 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Removing
> filesystem for file:///
> 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Removing
> filesystem for file:///
> 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Done
> clearing cache
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 55.501s
> [INFO] Finished at: Tue Aug 27 09:24:07 IST 2013
> [INFO] Final Memory: 36M/429M
> [INFO]
> ------------------------------------------------------------------------
>
>
> --
> Regards,
> Tanujit
>
Re: Error with ParquetRecordReaderTest
Posted by Jacques Nadeau <ja...@apache.org>.
We shouldn't be leaking but we're also pre-alpha... Maybe you could track
down where you think we should be releasing the resources?
On Mon, Aug 26, 2013 at 10:47 PM, Jacques Nadeau <ja...@apache.org> wrote:
> That looks like a Netty resource leak warning. Is there actually an
> exception or is that just the stack trace informing us of where we're
> allocating the leaked buffer?
>
> j
>
>
> On Mon, Aug 26, 2013 at 10:20 PM, Jason Altekruse <
> altekrusejason@gmail.com> wrote:
>
>> Well maven didn't report a failure, but the stack trace does indicate an
>> error occurred. It looks like we might be eating an exception with a print
>> of a stack trace.
>>
>> -Jason
>>
>>
>> On Tue, Aug 27, 2013 at 12:18 AM, Steven Phillips <sphillips@maprtech.com
>> >wrote:
>>
>> > Maybe I'm just not looking closely enough, but I don't see the problem.
>> It
>> > looks like the test passed without issue.
>> >
>> >
>> > On Mon, Aug 26, 2013 at 9:06 PM, Tanujit Ghosh <tanujit.ghosh@gmail.com
>> > >wrote:
>> >
>> > > Hi,
>> > >
>> > > I'm facing this issue when i try to run the test in
>> > >
>> > > org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
>> > >
>> > > [tanujit@legolas java-exec]$ mvn
>> > > -Dtest=ParquetRecordReaderTest#testMultipleRowGroupsAndReads test
>> > > [INFO] Scanning for projects...
>> > > [WARNING]
>> > > [WARNING] Some problems were encountered while building the effective
>> > model
>> > > for org.apache.drill.exec:java-exec:jar:1.0-SNAPSHOT
>> > > [WARNING] Detected profiles.xml alongside
>> > > org.apache.drill.exec:java-exec:1.0-SNAPSHOT, this file is no longer
>> > > supported and was ignored, please use the settings.xml instead
>> > > [WARNING]
>> > > [WARNING] It is highly recommended to fix these problems because they
>> > > threaten the stability of your build.
>> > > [WARNING]
>> > > [WARNING] For this reason, future Maven versions might no longer
>> support
>> > > building such malformed projects.
>> > > [WARNING]
>> > > [INFO]
>> > >
>> > > [INFO]
>> > >
>> ------------------------------------------------------------------------
>> > > [INFO] Building java-exec 1.0-SNAPSHOT
>> > > [INFO]
>> > >
>> ------------------------------------------------------------------------
>> > > Downloading:
>> > >
>> > >
>> >
>> http://repository.apache.org/snapshots/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
>> > > Downloading:
>> > >
>> > >
>> >
>> http://conjars.org/repo/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
>> > > Downloading:
>> > >
>> > >
>> >
>> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
>> > > Downloading:
>> > >
>> > >
>> >
>> http://repository.apache.org/snapshots/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
>> > > Downloading:
>> > >
>> > >
>> >
>> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
>> > > Downloading:
>> > >
>> > >
>> >
>> http://conjars.org/repo/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
>> > > Downloading:
>> > >
>> > >
>> >
>> http://repository.apache.org/snapshots/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
>> > > Downloading:
>> > >
>> > >
>> >
>> http://conjars.org/repo/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
>> > > Downloading:
>> > >
>> > >
>> >
>> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
>> > > [INFO]
>> > > [INFO] --- maven-resources-plugin:2.6:copy-resources (copy-resources)
>> @
>> > > java-exec ---
>> > > [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> > > [INFO] Copying 370 resources
>> > > [INFO]
>> > > [INFO] --- maven-enforcer-plugin:1.2:enforce (no_commons_logging) @
>> > > java-exec ---
>> > > [INFO]
>> > > [INFO] --- maven-antrun-plugin:1.6:run (generate-sources) @ java-exec
>> ---
>> > > [WARNING] Parameter tasks is deprecated, use target instead
>> > > [INFO] Executing tasks
>> > >
>> > > main:
>> > > [INFO] Executed tasks
>> > > [INFO] Registering compile source root
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/generated-sources
>> > > [INFO]
>> > > [INFO] --- fmpp-maven-plugin:1.0:generate (generate-sources) @
>> java-exec
>> > > ---
>> > > Downloading:
>> > >
>> > >
>> >
>> https://oss.sonatype.org/content/repositories/snapshots/org/freemarker/freemarker/maven-metadata.xml
>> > > Downloading:
>> > >
>> > >
>> >
>> https://oss.sonatype.org/content/repositories/snapshots/oro/oro/maven-metadata.xml
>> > > Downloading:
>> > >
>> > >
>> >
>> https://oss.sonatype.org/content/repositories/snapshots/org/beanshell/bsh/maven-metadata.xml
>> > > Downloading:
>> > >
>> > >
>> >
>> https://oss.sonatype.org/content/repositories/snapshots/xml-resolver/xml-resolver/maven-metadata.xml
>> > > - Executing: ValueHolders.java
>> > > log4j:WARN No appenders could be found for logger (freemarker.cache).
>> > > log4j:WARN Please initialize the log4j system properly.
>> > > - Executing: VariableLengthVectors.java
>> > > - Executing: FixedValueVectors.java
>> > > - Executing: TypeHelper.java
>> > > - Executing: NullableValueVectors.java
>> > > - Executing: RepeatedValueVectors.java
>> > > [INFO] Done
>> > > [INFO]
>> > > [INFO] --- maven-remote-resources-plugin:1.1:process (default) @
>> > java-exec
>> > > ---
>> > > [INFO] Setting property: classpath.resource.loader.class =>
>> > > 'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'.
>> > > [INFO] Setting property: velocimacro.messages.on => 'false'.
>> > > [INFO] Setting property: resource.loader => 'classpath'.
>> > > [INFO] Setting property: resource.manager.logwhenfound => 'false'.
>> > > [INFO]
>> > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
>> > > java-exec ---
>> > > [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> > > [INFO] Copying 1 resource
>> > > [INFO] Copying 3 resources
>> > > [INFO]
>> > > [INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @
>> > java-exec
>> > > ---
>> > > [INFO] Changes detected - recompiling the module!
>> > > [INFO] Compiling 460 source files to
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/classes
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetStorageEngine.java:[20,21]
>> > > com.sun.corba.se.impl.interceptors.CodecFactoryImpl is internal
>> > proprietary
>> > > API and may be removed in a future release
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/bit/BitConnection.java:
>> > > Some input files use or override a deprecated API.
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/bit/BitConnection.java:
>> > > Recompile with -Xlint:deprecation for details.
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java:
>> > > Some input files use unchecked or unsafe operations.
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java:
>> > > Recompile with -Xlint:unchecked for details.
>> > > [INFO]
>> > > [INFO] --- maven-resources-plugin:2.6:testResources
>> > (default-testResources)
>> > > @ java-exec ---
>> > > [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> > > [INFO] Copying 37 resources
>> > > [INFO] Copying 3 resources
>> > > [INFO]
>> > > [INFO] --- maven-compiler-plugin:3.0:testCompile
>> (default-testCompile) @
>> > > java-exec ---
>> > > [INFO] Changes detected - recompiling the module!
>> > > [INFO] Compiling 127 source files to
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/test-classes
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[25,16]
>> > > sun.misc.Unsafe is internal proprietary API and may be removed in a
>> > future
>> > > release
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[53,17]
>> > > sun.misc.Unsafe is internal proprietary API and may be removed in a
>> > future
>> > > release
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[94,12]
>> > > sun.misc.Unsafe is internal proprietary API and may be removed in a
>> > future
>> > > release
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[95,14]
>> > > sun.misc.Unsafe is internal proprietary API and may be removed in a
>> > future
>> > > release
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[98,26]
>> > > sun.misc.Unsafe is internal proprietary API and may be removed in a
>> > future
>> > > release
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[100,27]
>> > > sun.misc.Unsafe is internal proprietary API and may be removed in a
>> > future
>> > > release
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/TestParquetPhysicalPlan.java:
>> > > Some input files use or override a deprecated API.
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/TestParquetPhysicalPlan.java:
>> > > Recompile with -Xlint:deprecation for details.
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java:
>> > > Some input files use unchecked or unsafe operations.
>> > > [WARNING]
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java:
>> > > Recompile with -Xlint:unchecked for details.
>> > > [INFO]
>> > > [INFO] --- maven-surefire-plugin:2.15:test (default-test) @ java-exec
>> ---
>> > > [INFO] Surefire report directory:
>> > >
>> > >
>> >
>> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/surefire-reports
>> > >
>> > > -------------------------------------------------------
>> > > T E S T S
>> > > -------------------------------------------------------
>> > > Running org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
>> > > 09:23:52.569 [main] DEBUG org.apache.hadoop.security.Groups -
>> Creating
>> > new
>> > > Groups object
>> > > 09:23:52.662 [main] DEBUG org.apache.hadoop.security.Groups - Group
>> > mapping
>> > > impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping;
>> > > cacheTimeout=300000
>> > > 09:23:52.841 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop
>> > > login
>> > > 09:23:52.842 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop
>> > > login commit
>> > > 09:23:52.863 [main] DEBUG o.a.h.security.UserGroupInformation - using
>> > local
>> > > user:UnixPrincipal: tanujit
>> > > 09:23:52.867 [main] DEBUG o.a.h.security.UserGroupInformation - UGI
>> > > loginUser:tanujit
>> > > 09:23:52.921 [main] DEBUG org.apache.hadoop.fs.FileSystem - Creating
>> > > filesystem for file:///
>> > > 09:23:53.011 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - Trying to
>> > load
>> > > the custom-built native-hadoop library...
>> > > 09:23:53.033 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - Failed to
>> > load
>> > > native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in
>> > > java.library.path
>> > > 09:23:53.034 [main] DEBUG o.a.hadoop.util.NativeCodeLoader -
>> > >
>> > >
>> >
>> java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>> > > 09:23:53.034 [main] WARN o.a.hadoop.util.NativeCodeLoader - Unable to
>> > load
>> > > native-hadoop library for your platform... using builtin-java classes
>> > where
>> > > applicable
>> > > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
>> > > SLF4J: Defaulting to no-operation (NOP) logger implementation
>> > > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
>> > further
>> > > details.
>> > > 09:23:55.443 [main] DEBUG org.reflections.Reflections - going to scan
>> > these
>> > > urls:
>> > >
>> > >
>> >
>> jar:file:/home/tanujit/.m2/repository/org/apache/drill/common/1.0-SNAPSHOT/common-1.0-SNAPSHOT.jar!/
>> > >
>> > >
>> >
>> file:/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/test-classes/
>> > >
>> > >
>> >
>> file:/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/classes/
>> > >
>> > >
>> >
>> jar:file:/home/tanujit/.m2/repository/org/apache/drill/common/1.0-SNAPSHOT/common-1.0-SNAPSHOT-tests.jar!/
>> > >
>> > > 09:23:56.000 [main] INFO org.reflections.Reflections - Reflections
>> took
>> > > 556 ms to scan 4 urls, producing 669 keys and 1634 values
>> > > 09:23:56.028 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using
>> SLF4J
>> > as
>> > > the default logging framework
>> > > 09:23:56.037 [main] DEBUG i.n.c.MultithreadEventLoopGroup -
>> > > -Dio.netty.eventLoopThreads: 8
>> > > 09:23:56.074 [main] DEBUG i.n.util.internal.PlatformDependent - UID:
>> 1000
>> > > 09:23:56.075 [main] DEBUG i.n.util.internal.PlatformDependent - Java
>> > > version: 7
>> > > 09:23:56.076 [main] DEBUG i.n.util.internal.PlatformDependent -
>> > > -Dio.netty.noUnsafe: false
>> > > 09:23:56.079 [main] DEBUG i.n.util.internal.PlatformDependent0 -
>> > > java.nio.ByteBuffer.cleaner: available
>> > > 09:23:56.079 [main] DEBUG i.n.util.internal.PlatformDependent0 -
>> > > java.nio.Buffer.address: available
>> > > 09:23:56.080 [main] DEBUG i.n.util.internal.PlatformDependent0 -
>> > > sun.misc.Unsafe.theUnsafe: available
>> > > 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent0 -
>> > > sun.misc.Unsafe.copyMemory: available
>> > > 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent0 -
>> > > java.nio.Bits.unaligned: true
>> > > 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent -
>> > > sun.misc.Unsafe: available
>> > > 09:23:56.082 [main] DEBUG i.n.util.internal.PlatformDependent -
>> > > -Dio.netty.noJavassist: false
>> > > 09:23:56.145 [main] DEBUG i.n.util.internal.PlatformDependent -
>> > Javassist:
>> > > available
>> > > 09:23:56.145 [main] DEBUG i.n.util.internal.PlatformDependent -
>> > > -Dio.netty.noPreferDirect: false
>> > > 09:23:56.164 [main] DEBUG io.netty.channel.nio.NioEventLoop -
>> > > -Dio.netty.noKeySetOptimization: false
>> > > 09:23:56.164 [main] DEBUG io.netty.channel.nio.NioEventLoop -
>> > > -Dio.netty.selectorAutoRebuildThreshold: 512
>> > > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
>> > > -Dio.netty.allocator.numHeapArenas: 4
>> > > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
>> > > -Dio.netty.allocator.numDirectArenas: 4
>> > > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
>> > > -Dio.netty.allocator.pageSize: 8192
>> > > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
>> > > -Dio.netty.allocator.maxOrder: 11
>> > > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
>> > > -Dio.netty.allocator.chunkSize: 16777216
>> > > running bit
>> > > 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback interface:
>> lo
>> > > 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback address:
>> > > /0:0:0:0:0:0:0:1%1 (primary)
>> > > 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback address: /
>> > > 127.0.0.1
>> > > 09:23:56.277 [main] DEBUG io.netty.util.NetUtil -
>> > > /proc/sys/net/core/somaxconn: 128
>> > > 09:23:56.288 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
>> > [id:
>> > > 0x036be036] REGISTERED
>> > > 09:23:56.290 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
>> > [id:
>> > > 0x036be036] BIND(0.0.0.0/0.0.0.0:31010)
>> > > 09:23:56.294 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
>> > [id:
>> > > 0x036be036, /0:0:0:0:0:0:0:0:31010] ACTIVE
>> > > 09:23:56.310 [BitServer-1] INFO i.n.handler.logging.LoggingHandler -
>> > [id:
>> > > 0xb8299b35] REGISTERED
>> > > 09:23:56.311 [BitServer-1] INFO i.n.handler.logging.LoggingHandler -
>> > [id:
>> > > 0xb8299b35] BIND(0.0.0.0/0.0.0.0:31011)
>> > > 09:23:56.311 [BitServer-1] INFO i.n.handler.logging.LoggingHandler -
>> > [id:
>> > > 0xb8299b35, /0:0:0:0:0:0:0:0:31011] ACTIVE
>> > > running client
>> > > connecting - legolas -- 31010
>> > > 09:23:56.545 [Client-1] DEBUG
>> > > i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
>> > > io.netty.util.internal.__matchers__.io.netty.buffer.ByteBufMatcher
>> > > 09:23:56.549 [Client-1] DEBUG
>> > > i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
>> > >
>> > >
>> >
>> io.netty.util.internal.__matchers__.org.apache.drill.exec.rpc.OutboundRpcMessageMatcher
>> > > 09:23:56.550 [Client-1] DEBUG
>> > > i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
>> > >
>> > >
>> >
>> io.netty.util.internal.__matchers__.org.apache.drill.exec.rpc.InboundRpcMessageMatcher
>> > > 09:23:56.554 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
>> > [id:
>> > > 0x036be036, /0:0:0:0:0:0:0:0:31010] RECEIVED: [id: 0xf046bccd, /
>> > > 192.168.0.103:32952 => /192.168.0.103:31010]
>> > > 09:23:56.568 [Client-1] DEBUG io.netty.util.ResourceLeakDetector -
>> > > -Dio.netty.noResourceLeakDetection: false
>> > > running the query
>> > > Screen [endpoint=address: "legolas"
>> > > user_port: 31010
>> > > bit_port: 31011
>> > > ,
>> > getChild()=org.apache.drill.exec.store.parquet.ParquetGroupScan@59c81460
>> > > ]
>> > > 09:24:00.420 [WorkManager-2] WARN io.netty.util.ResourceLeakDetector
>> -
>> > > LEAK: ByteBuf was GC'd before being released correctly. The following
>> > > stack trace shows where the leaked object was created, rather than
>> where
>> > > you failed to release it.
>> > > io.netty.util.ResourceLeakException:
>> > > io.netty.buffer.PooledUnsafeDirectByteBufL@659974ba
>> > > at
>> > >
>> > >
>> >
>> io.netty.util.ResourceLeakDetector$DefaultResourceLeak.<init>(ResourceLeakDetector.java:174)
>> > > ~[netty-common-4.0.7.Final.jar:na]
>> > > at
>> io.netty.util.ResourceLeakDetector.open(ResourceLeakDetector.java:116)
>> > > ~[netty-common-4.0.7.Final.jar:na]
>> > > at io.netty.buffer.PooledByteBufL.<init>(PooledByteBufL.java:45)
>> > > ~[netty-bufferl-4.0.7.Final.jar:na]
>> > > at
>> > >
>> > >
>> >
>> io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:53)
>> > > ~[netty-bufferl-4.0.7.Final.jar:na]
>> > > at
>> > >
>> > >
>> >
>> io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:32)
>> > > ~[netty-bufferl-4.0.7.Final.jar:na]
>> > > at
>> > >
>> > >
>> >
>> io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:39)
>> > > ~[netty-bufferl-4.0.7.Final.jar:na]
>> > > at
>> > >
>> > >
>> >
>> io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:36)
>> > > ~[netty-bufferl-4.0.7.Final.jar:na]
>> > > at io.netty.util.Recycler.get(Recycler.java:40)
>> > > ~[netty-common-4.0.7.Final.jar:na]
>> > > at
>> > >
>> > >
>> >
>> io.netty.buffer.PooledUnsafeDirectByteBufL.newInstance(PooledUnsafeDirectByteBufL.java:44)
>> > > ~[netty-bufferl-4.0.7.Final.jar:na]
>> > > at
>> io.netty.buffer.PoolArenaL$DirectArena.newByteBuf(PoolArenaL.java:397)
>> > > ~[netty-bufferl-4.0.7.Final.jar:na]
>> > > at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:93)
>> > > ~[netty-bufferl-4.0.7.Final.jar:na]
>> > > at
>> > >
>> > >
>> >
>> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:235)
>> > > ~[netty-bufferl-4.0.7.Final.jar:na]
>> > > at
>> > >
>> > >
>> >
>> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132)
>> > > ~[netty-buffer-4.0.7.Final.jar:na]
>> > > at
>> > >
>> > >
>> >
>> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123)
>> > > ~[netty-buffer-4.0.7.Final.jar:na]
>> > > at
>> > >
>> > >
>> >
>> org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35)
>> > > ~[classes/:na]
>> > > at
>> org.apache.drill.exec.vector.BitVector.allocateNew(BitVector.java:54)
>> > > ~[classes/:na]
>> > > at
>> > >
>> > >
>> >
>> org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:12)
>> > > ~[classes/:na]
>> > > at
>> > >
>> >
>> org.apache.drill.exec.store.VectorHolder.allocateNew(VectorHolder.java:87)
>> > > ~[classes/:na]
>> > > at
>> org.apache.drill.exec.store.VectorHolder.reset(VectorHolder.java:74)
>> > > ~[classes/:na]
>> > > at
>> > >
>> > >
>> >
>> org.apache.drill.exec.store.parquet.ParquetRecordReader.resetBatch(ParquetRecordReader.java:276)
>> > > ~[classes/:na]
>> > > at
>> > >
>> > >
>> >
>> org.apache.drill.exec.store.parquet.ParquetRecordReader.next(ParquetRecordReader.java:318)
>> > > ~[classes/:na]
>> > > at
>> org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:99)
>> > > ~[classes/:na]
>> > > at
>> > >
>> > >
>> >
>> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:77)
>> > > ~[classes/:na]
>> > > at
>> org.apache.drill.exec.work.FragmentRunner.run(FragmentRunner.java:79)
>> > > ~[classes/:na]
>> > > at
>> > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> > > ~[na:1.7.0_25]
>> > > at
>> > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> > > ~[na:1.7.0_25]
>> > > at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
>> > > Took 10003 ms to run query
>> > > 09:24:07.614 [BitServer-1] INFO i.n.handler.logging.LoggingHandler -
>> > [id:
>> > > 0xb8299b35, /0:0:0:0:0:0:0:0:31011] UNREGISTERED
>> > > 09:24:07.614 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
>> > [id:
>> > > 0x036be036, /0:0:0:0:0:0:0:0:31010] UNREGISTERED
>> > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.329
>> > sec
>> > > - in org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
>> > > 09:24:07.621 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem -
>> Starting
>> > > clear of FileSystem cache with 1 elements.
>> > > 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem -
>> Removing
>> > > filesystem for file:///
>> > > 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem -
>> Removing
>> > > filesystem for file:///
>> > > 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Done
>> > > clearing cache
>> > >
>> > > Results :
>> > >
>> > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>> > >
>> > > [INFO]
>> > >
>> ------------------------------------------------------------------------
>> > > [INFO] BUILD SUCCESS
>> > > [INFO]
>> > >
>> ------------------------------------------------------------------------
>> > > [INFO] Total time: 55.501s
>> > > [INFO] Finished at: Tue Aug 27 09:24:07 IST 2013
>> > > [INFO] Final Memory: 36M/429M
>> > > [INFO]
>> > >
>> ------------------------------------------------------------------------
>> > >
>> > >
>> > > --
>> > > Regards,
>> > > Tanujit
>> > >
>> >
>>
>
>
Re: Error with ParquetRecordReaderTest
Posted by Jacques Nadeau <ja...@apache.org>.
That looks like a Netty resource leak warning. Is there actually an
exception or is that just the stack trace informing us of where we're
allocating the leaked buffer?
j
On Mon, Aug 26, 2013 at 10:20 PM, Jason Altekruse
<al...@gmail.com>wrote:
> Well maven didn't report a failure, but the stack trace does indicate an
> error occurred. It looks like we might be eating an exception with a print
> of a stack trace.
>
> -Jason
>
>
> On Tue, Aug 27, 2013 at 12:18 AM, Steven Phillips <sphillips@maprtech.com
> >wrote:
>
> > Maybe I'm just not looking closely enough, but I don't see the problem.
> It
> > looks like the test passed without issue.
> >
> >
> > On Mon, Aug 26, 2013 at 9:06 PM, Tanujit Ghosh <tanujit.ghosh@gmail.com
> > >wrote:
> >
> > > Hi,
> > >
> > > I'm facing this issue when i try to run the test in
> > >
> > > org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
> > >
> > > [tanujit@legolas java-exec]$ mvn
> > > -Dtest=ParquetRecordReaderTest#testMultipleRowGroupsAndReads test
> > > [INFO] Scanning for projects...
> > > [WARNING]
> > > [WARNING] Some problems were encountered while building the effective
> > model
> > > for org.apache.drill.exec:java-exec:jar:1.0-SNAPSHOT
> > > [WARNING] Detected profiles.xml alongside
> > > org.apache.drill.exec:java-exec:1.0-SNAPSHOT, this file is no longer
> > > supported and was ignored, please use the settings.xml instead
> > > [WARNING]
> > > [WARNING] It is highly recommended to fix these problems because they
> > > threaten the stability of your build.
> > > [WARNING]
> > > [WARNING] For this reason, future Maven versions might no longer
> support
> > > building such malformed projects.
> > > [WARNING]
> > > [INFO]
> > >
> > > [INFO]
> > >
> ------------------------------------------------------------------------
> > > [INFO] Building java-exec 1.0-SNAPSHOT
> > > [INFO]
> > >
> ------------------------------------------------------------------------
> > > Downloading:
> > >
> > >
> >
> http://repository.apache.org/snapshots/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
> > > Downloading:
> > >
> > >
> >
> http://conjars.org/repo/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
> > > Downloading:
> > >
> > >
> >
> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
> > > Downloading:
> > >
> > >
> >
> http://repository.apache.org/snapshots/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
> > > Downloading:
> > >
> > >
> >
> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
> > > Downloading:
> > >
> > >
> >
> http://conjars.org/repo/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
> > > Downloading:
> > >
> > >
> >
> http://repository.apache.org/snapshots/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
> > > Downloading:
> > >
> > >
> >
> http://conjars.org/repo/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
> > > Downloading:
> > >
> > >
> >
> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
> > > [INFO]
> > > [INFO] --- maven-resources-plugin:2.6:copy-resources (copy-resources) @
> > > java-exec ---
> > > [INFO] Using 'UTF-8' encoding to copy filtered resources.
> > > [INFO] Copying 370 resources
> > > [INFO]
> > > [INFO] --- maven-enforcer-plugin:1.2:enforce (no_commons_logging) @
> > > java-exec ---
> > > [INFO]
> > > [INFO] --- maven-antrun-plugin:1.6:run (generate-sources) @ java-exec
> ---
> > > [WARNING] Parameter tasks is deprecated, use target instead
> > > [INFO] Executing tasks
> > >
> > > main:
> > > [INFO] Executed tasks
> > > [INFO] Registering compile source root
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/generated-sources
> > > [INFO]
> > > [INFO] --- fmpp-maven-plugin:1.0:generate (generate-sources) @
> java-exec
> > > ---
> > > Downloading:
> > >
> > >
> >
> https://oss.sonatype.org/content/repositories/snapshots/org/freemarker/freemarker/maven-metadata.xml
> > > Downloading:
> > >
> > >
> >
> https://oss.sonatype.org/content/repositories/snapshots/oro/oro/maven-metadata.xml
> > > Downloading:
> > >
> > >
> >
> https://oss.sonatype.org/content/repositories/snapshots/org/beanshell/bsh/maven-metadata.xml
> > > Downloading:
> > >
> > >
> >
> https://oss.sonatype.org/content/repositories/snapshots/xml-resolver/xml-resolver/maven-metadata.xml
> > > - Executing: ValueHolders.java
> > > log4j:WARN No appenders could be found for logger (freemarker.cache).
> > > log4j:WARN Please initialize the log4j system properly.
> > > - Executing: VariableLengthVectors.java
> > > - Executing: FixedValueVectors.java
> > > - Executing: TypeHelper.java
> > > - Executing: NullableValueVectors.java
> > > - Executing: RepeatedValueVectors.java
> > > [INFO] Done
> > > [INFO]
> > > [INFO] --- maven-remote-resources-plugin:1.1:process (default) @
> > java-exec
> > > ---
> > > [INFO] Setting property: classpath.resource.loader.class =>
> > > 'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'.
> > > [INFO] Setting property: velocimacro.messages.on => 'false'.
> > > [INFO] Setting property: resource.loader => 'classpath'.
> > > [INFO] Setting property: resource.manager.logwhenfound => 'false'.
> > > [INFO]
> > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
> > > java-exec ---
> > > [INFO] Using 'UTF-8' encoding to copy filtered resources.
> > > [INFO] Copying 1 resource
> > > [INFO] Copying 3 resources
> > > [INFO]
> > > [INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @
> > java-exec
> > > ---
> > > [INFO] Changes detected - recompiling the module!
> > > [INFO] Compiling 460 source files to
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/classes
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetStorageEngine.java:[20,21]
> > > com.sun.corba.se.impl.interceptors.CodecFactoryImpl is internal
> > proprietary
> > > API and may be removed in a future release
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/bit/BitConnection.java:
> > > Some input files use or override a deprecated API.
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/bit/BitConnection.java:
> > > Recompile with -Xlint:deprecation for details.
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java:
> > > Some input files use unchecked or unsafe operations.
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java:
> > > Recompile with -Xlint:unchecked for details.
> > > [INFO]
> > > [INFO] --- maven-resources-plugin:2.6:testResources
> > (default-testResources)
> > > @ java-exec ---
> > > [INFO] Using 'UTF-8' encoding to copy filtered resources.
> > > [INFO] Copying 37 resources
> > > [INFO] Copying 3 resources
> > > [INFO]
> > > [INFO] --- maven-compiler-plugin:3.0:testCompile (default-testCompile)
> @
> > > java-exec ---
> > > [INFO] Changes detected - recompiling the module!
> > > [INFO] Compiling 127 source files to
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/test-classes
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[25,16]
> > > sun.misc.Unsafe is internal proprietary API and may be removed in a
> > future
> > > release
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[53,17]
> > > sun.misc.Unsafe is internal proprietary API and may be removed in a
> > future
> > > release
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[94,12]
> > > sun.misc.Unsafe is internal proprietary API and may be removed in a
> > future
> > > release
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[95,14]
> > > sun.misc.Unsafe is internal proprietary API and may be removed in a
> > future
> > > release
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[98,26]
> > > sun.misc.Unsafe is internal proprietary API and may be removed in a
> > future
> > > release
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[100,27]
> > > sun.misc.Unsafe is internal proprietary API and may be removed in a
> > future
> > > release
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/TestParquetPhysicalPlan.java:
> > > Some input files use or override a deprecated API.
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/TestParquetPhysicalPlan.java:
> > > Recompile with -Xlint:deprecation for details.
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java:
> > > Some input files use unchecked or unsafe operations.
> > > [WARNING]
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java:
> > > Recompile with -Xlint:unchecked for details.
> > > [INFO]
> > > [INFO] --- maven-surefire-plugin:2.15:test (default-test) @ java-exec
> ---
> > > [INFO] Surefire report directory:
> > >
> > >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/surefire-reports
> > >
> > > -------------------------------------------------------
> > > T E S T S
> > > -------------------------------------------------------
> > > Running org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
> > > 09:23:52.569 [main] DEBUG org.apache.hadoop.security.Groups - Creating
> > new
> > > Groups object
> > > 09:23:52.662 [main] DEBUG org.apache.hadoop.security.Groups - Group
> > mapping
> > > impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping;
> > > cacheTimeout=300000
> > > 09:23:52.841 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop
> > > login
> > > 09:23:52.842 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop
> > > login commit
> > > 09:23:52.863 [main] DEBUG o.a.h.security.UserGroupInformation - using
> > local
> > > user:UnixPrincipal: tanujit
> > > 09:23:52.867 [main] DEBUG o.a.h.security.UserGroupInformation - UGI
> > > loginUser:tanujit
> > > 09:23:52.921 [main] DEBUG org.apache.hadoop.fs.FileSystem - Creating
> > > filesystem for file:///
> > > 09:23:53.011 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - Trying to
> > load
> > > the custom-built native-hadoop library...
> > > 09:23:53.033 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - Failed to
> > load
> > > native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in
> > > java.library.path
> > > 09:23:53.034 [main] DEBUG o.a.hadoop.util.NativeCodeLoader -
> > >
> > >
> >
> java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > > 09:23:53.034 [main] WARN o.a.hadoop.util.NativeCodeLoader - Unable to
> > load
> > > native-hadoop library for your platform... using builtin-java classes
> > where
> > > applicable
> > > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> > > SLF4J: Defaulting to no-operation (NOP) logger implementation
> > > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
> > further
> > > details.
> > > 09:23:55.443 [main] DEBUG org.reflections.Reflections - going to scan
> > these
> > > urls:
> > >
> > >
> >
> jar:file:/home/tanujit/.m2/repository/org/apache/drill/common/1.0-SNAPSHOT/common-1.0-SNAPSHOT.jar!/
> > >
> > >
> >
> file:/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/test-classes/
> > >
> > >
> >
> file:/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/classes/
> > >
> > >
> >
> jar:file:/home/tanujit/.m2/repository/org/apache/drill/common/1.0-SNAPSHOT/common-1.0-SNAPSHOT-tests.jar!/
> > >
> > > 09:23:56.000 [main] INFO org.reflections.Reflections - Reflections
> took
> > > 556 ms to scan 4 urls, producing 669 keys and 1634 values
> > > 09:23:56.028 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J
> > as
> > > the default logging framework
> > > 09:23:56.037 [main] DEBUG i.n.c.MultithreadEventLoopGroup -
> > > -Dio.netty.eventLoopThreads: 8
> > > 09:23:56.074 [main] DEBUG i.n.util.internal.PlatformDependent - UID:
> 1000
> > > 09:23:56.075 [main] DEBUG i.n.util.internal.PlatformDependent - Java
> > > version: 7
> > > 09:23:56.076 [main] DEBUG i.n.util.internal.PlatformDependent -
> > > -Dio.netty.noUnsafe: false
> > > 09:23:56.079 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> > > java.nio.ByteBuffer.cleaner: available
> > > 09:23:56.079 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> > > java.nio.Buffer.address: available
> > > 09:23:56.080 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> > > sun.misc.Unsafe.theUnsafe: available
> > > 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> > > sun.misc.Unsafe.copyMemory: available
> > > 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> > > java.nio.Bits.unaligned: true
> > > 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent -
> > > sun.misc.Unsafe: available
> > > 09:23:56.082 [main] DEBUG i.n.util.internal.PlatformDependent -
> > > -Dio.netty.noJavassist: false
> > > 09:23:56.145 [main] DEBUG i.n.util.internal.PlatformDependent -
> > Javassist:
> > > available
> > > 09:23:56.145 [main] DEBUG i.n.util.internal.PlatformDependent -
> > > -Dio.netty.noPreferDirect: false
> > > 09:23:56.164 [main] DEBUG io.netty.channel.nio.NioEventLoop -
> > > -Dio.netty.noKeySetOptimization: false
> > > 09:23:56.164 [main] DEBUG io.netty.channel.nio.NioEventLoop -
> > > -Dio.netty.selectorAutoRebuildThreshold: 512
> > > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> > > -Dio.netty.allocator.numHeapArenas: 4
> > > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> > > -Dio.netty.allocator.numDirectArenas: 4
> > > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> > > -Dio.netty.allocator.pageSize: 8192
> > > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> > > -Dio.netty.allocator.maxOrder: 11
> > > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> > > -Dio.netty.allocator.chunkSize: 16777216
> > > running bit
> > > 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback interface:
> lo
> > > 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback address:
> > > /0:0:0:0:0:0:0:1%1 (primary)
> > > 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback address: /
> > > 127.0.0.1
> > > 09:23:56.277 [main] DEBUG io.netty.util.NetUtil -
> > > /proc/sys/net/core/somaxconn: 128
> > > 09:23:56.288 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
> > [id:
> > > 0x036be036] REGISTERED
> > > 09:23:56.290 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
> > [id:
> > > 0x036be036] BIND(0.0.0.0/0.0.0.0:31010)
> > > 09:23:56.294 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
> > [id:
> > > 0x036be036, /0:0:0:0:0:0:0:0:31010] ACTIVE
> > > 09:23:56.310 [BitServer-1] INFO i.n.handler.logging.LoggingHandler -
> > [id:
> > > 0xb8299b35] REGISTERED
> > > 09:23:56.311 [BitServer-1] INFO i.n.handler.logging.LoggingHandler -
> > [id:
> > > 0xb8299b35] BIND(0.0.0.0/0.0.0.0:31011)
> > > 09:23:56.311 [BitServer-1] INFO i.n.handler.logging.LoggingHandler -
> > [id:
> > > 0xb8299b35, /0:0:0:0:0:0:0:0:31011] ACTIVE
> > > running client
> > > connecting - legolas -- 31010
> > > 09:23:56.545 [Client-1] DEBUG
> > > i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
> > > io.netty.util.internal.__matchers__.io.netty.buffer.ByteBufMatcher
> > > 09:23:56.549 [Client-1] DEBUG
> > > i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
> > >
> > >
> >
> io.netty.util.internal.__matchers__.org.apache.drill.exec.rpc.OutboundRpcMessageMatcher
> > > 09:23:56.550 [Client-1] DEBUG
> > > i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
> > >
> > >
> >
> io.netty.util.internal.__matchers__.org.apache.drill.exec.rpc.InboundRpcMessageMatcher
> > > 09:23:56.554 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
> > [id:
> > > 0x036be036, /0:0:0:0:0:0:0:0:31010] RECEIVED: [id: 0xf046bccd, /
> > > 192.168.0.103:32952 => /192.168.0.103:31010]
> > > 09:23:56.568 [Client-1] DEBUG io.netty.util.ResourceLeakDetector -
> > > -Dio.netty.noResourceLeakDetection: false
> > > running the query
> > > Screen [endpoint=address: "legolas"
> > > user_port: 31010
> > > bit_port: 31011
> > > ,
> > getChild()=org.apache.drill.exec.store.parquet.ParquetGroupScan@59c81460
> > > ]
> > > 09:24:00.420 [WorkManager-2] WARN io.netty.util.ResourceLeakDetector -
> > > LEAK: ByteBuf was GC'd before being released correctly. The following
> > > stack trace shows where the leaked object was created, rather than
> where
> > > you failed to release it.
> > > io.netty.util.ResourceLeakException:
> > > io.netty.buffer.PooledUnsafeDirectByteBufL@659974ba
> > > at
> > >
> > >
> >
> io.netty.util.ResourceLeakDetector$DefaultResourceLeak.<init>(ResourceLeakDetector.java:174)
> > > ~[netty-common-4.0.7.Final.jar:na]
> > > at
> io.netty.util.ResourceLeakDetector.open(ResourceLeakDetector.java:116)
> > > ~[netty-common-4.0.7.Final.jar:na]
> > > at io.netty.buffer.PooledByteBufL.<init>(PooledByteBufL.java:45)
> > > ~[netty-bufferl-4.0.7.Final.jar:na]
> > > at
> > >
> > >
> >
> io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:53)
> > > ~[netty-bufferl-4.0.7.Final.jar:na]
> > > at
> > >
> > >
> >
> io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:32)
> > > ~[netty-bufferl-4.0.7.Final.jar:na]
> > > at
> > >
> > >
> >
> io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:39)
> > > ~[netty-bufferl-4.0.7.Final.jar:na]
> > > at
> > >
> > >
> >
> io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:36)
> > > ~[netty-bufferl-4.0.7.Final.jar:na]
> > > at io.netty.util.Recycler.get(Recycler.java:40)
> > > ~[netty-common-4.0.7.Final.jar:na]
> > > at
> > >
> > >
> >
> io.netty.buffer.PooledUnsafeDirectByteBufL.newInstance(PooledUnsafeDirectByteBufL.java:44)
> > > ~[netty-bufferl-4.0.7.Final.jar:na]
> > > at
> io.netty.buffer.PoolArenaL$DirectArena.newByteBuf(PoolArenaL.java:397)
> > > ~[netty-bufferl-4.0.7.Final.jar:na]
> > > at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:93)
> > > ~[netty-bufferl-4.0.7.Final.jar:na]
> > > at
> > >
> > >
> >
> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:235)
> > > ~[netty-bufferl-4.0.7.Final.jar:na]
> > > at
> > >
> > >
> >
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132)
> > > ~[netty-buffer-4.0.7.Final.jar:na]
> > > at
> > >
> > >
> >
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123)
> > > ~[netty-buffer-4.0.7.Final.jar:na]
> > > at
> > >
> > >
> >
> org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35)
> > > ~[classes/:na]
> > > at
> org.apache.drill.exec.vector.BitVector.allocateNew(BitVector.java:54)
> > > ~[classes/:na]
> > > at
> > >
> > >
> >
> org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:12)
> > > ~[classes/:na]
> > > at
> > >
> >
> org.apache.drill.exec.store.VectorHolder.allocateNew(VectorHolder.java:87)
> > > ~[classes/:na]
> > > at org.apache.drill.exec.store.VectorHolder.reset(VectorHolder.java:74)
> > > ~[classes/:na]
> > > at
> > >
> > >
> >
> org.apache.drill.exec.store.parquet.ParquetRecordReader.resetBatch(ParquetRecordReader.java:276)
> > > ~[classes/:na]
> > > at
> > >
> > >
> >
> org.apache.drill.exec.store.parquet.ParquetRecordReader.next(ParquetRecordReader.java:318)
> > > ~[classes/:na]
> > > at
> org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:99)
> > > ~[classes/:na]
> > > at
> > >
> > >
> >
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:77)
> > > ~[classes/:na]
> > > at
> org.apache.drill.exec.work.FragmentRunner.run(FragmentRunner.java:79)
> > > ~[classes/:na]
> > > at
> > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > > ~[na:1.7.0_25]
> > > at
> > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > > ~[na:1.7.0_25]
> > > at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
> > > Took 10003 ms to run query
> > > 09:24:07.614 [BitServer-1] INFO i.n.handler.logging.LoggingHandler -
> > [id:
> > > 0xb8299b35, /0:0:0:0:0:0:0:0:31011] UNREGISTERED
> > > 09:24:07.614 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
> > [id:
> > > 0x036be036, /0:0:0:0:0:0:0:0:31010] UNREGISTERED
> > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.329
> > sec
> > > - in org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
> > > 09:24:07.621 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem -
> Starting
> > > clear of FileSystem cache with 1 elements.
> > > 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem -
> Removing
> > > filesystem for file:///
> > > 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem -
> Removing
> > > filesystem for file:///
> > > 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Done
> > > clearing cache
> > >
> > > Results :
> > >
> > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> > >
> > > [INFO]
> > >
> ------------------------------------------------------------------------
> > > [INFO] BUILD SUCCESS
> > > [INFO]
> > >
> ------------------------------------------------------------------------
> > > [INFO] Total time: 55.501s
> > > [INFO] Finished at: Tue Aug 27 09:24:07 IST 2013
> > > [INFO] Final Memory: 36M/429M
> > > [INFO]
> > >
> ------------------------------------------------------------------------
> > >
> > >
> > > --
> > > Regards,
> > > Tanujit
> > >
> >
>
Re: Error with ParquetRecordReaderTest
Posted by Jason Altekruse <al...@gmail.com>.
Well maven didn't report a failure, but the stack trace does indicate an
error occurred. It looks like we might be eating an exception with a print
of a stack trace.
-Jason
On Tue, Aug 27, 2013 at 12:18 AM, Steven Phillips <sp...@maprtech.com>wrote:
> Maybe I'm just not looking closely enough, but I don't see the problem. It
> looks like the test passed without issue.
>
>
> On Mon, Aug 26, 2013 at 9:06 PM, Tanujit Ghosh <tanujit.ghosh@gmail.com
> >wrote:
>
> > Hi,
> >
> > I'm facing this issue when i try to run the test in
> >
> > org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
> >
> > [tanujit@legolas java-exec]$ mvn
> > -Dtest=ParquetRecordReaderTest#testMultipleRowGroupsAndReads test
> > [INFO] Scanning for projects...
> > [WARNING]
> > [WARNING] Some problems were encountered while building the effective
> model
> > for org.apache.drill.exec:java-exec:jar:1.0-SNAPSHOT
> > [WARNING] Detected profiles.xml alongside
> > org.apache.drill.exec:java-exec:1.0-SNAPSHOT, this file is no longer
> > supported and was ignored, please use the settings.xml instead
> > [WARNING]
> > [WARNING] It is highly recommended to fix these problems because they
> > threaten the stability of your build.
> > [WARNING]
> > [WARNING] For this reason, future Maven versions might no longer support
> > building such malformed projects.
> > [WARNING]
> > [INFO]
> >
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Building java-exec 1.0-SNAPSHOT
> > [INFO]
> > ------------------------------------------------------------------------
> > Downloading:
> >
> >
> http://repository.apache.org/snapshots/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
> > Downloading:
> >
> >
> http://conjars.org/repo/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
> > Downloading:
> >
> >
> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
> > Downloading:
> >
> >
> http://repository.apache.org/snapshots/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
> > Downloading:
> >
> >
> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
> > Downloading:
> >
> >
> http://conjars.org/repo/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
> > Downloading:
> >
> >
> http://repository.apache.org/snapshots/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
> > Downloading:
> >
> >
> http://conjars.org/repo/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
> > Downloading:
> >
> >
> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
> > [INFO]
> > [INFO] --- maven-resources-plugin:2.6:copy-resources (copy-resources) @
> > java-exec ---
> > [INFO] Using 'UTF-8' encoding to copy filtered resources.
> > [INFO] Copying 370 resources
> > [INFO]
> > [INFO] --- maven-enforcer-plugin:1.2:enforce (no_commons_logging) @
> > java-exec ---
> > [INFO]
> > [INFO] --- maven-antrun-plugin:1.6:run (generate-sources) @ java-exec ---
> > [WARNING] Parameter tasks is deprecated, use target instead
> > [INFO] Executing tasks
> >
> > main:
> > [INFO] Executed tasks
> > [INFO] Registering compile source root
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/generated-sources
> > [INFO]
> > [INFO] --- fmpp-maven-plugin:1.0:generate (generate-sources) @ java-exec
> > ---
> > Downloading:
> >
> >
> https://oss.sonatype.org/content/repositories/snapshots/org/freemarker/freemarker/maven-metadata.xml
> > Downloading:
> >
> >
> https://oss.sonatype.org/content/repositories/snapshots/oro/oro/maven-metadata.xml
> > Downloading:
> >
> >
> https://oss.sonatype.org/content/repositories/snapshots/org/beanshell/bsh/maven-metadata.xml
> > Downloading:
> >
> >
> https://oss.sonatype.org/content/repositories/snapshots/xml-resolver/xml-resolver/maven-metadata.xml
> > - Executing: ValueHolders.java
> > log4j:WARN No appenders could be found for logger (freemarker.cache).
> > log4j:WARN Please initialize the log4j system properly.
> > - Executing: VariableLengthVectors.java
> > - Executing: FixedValueVectors.java
> > - Executing: TypeHelper.java
> > - Executing: NullableValueVectors.java
> > - Executing: RepeatedValueVectors.java
> > [INFO] Done
> > [INFO]
> > [INFO] --- maven-remote-resources-plugin:1.1:process (default) @
> java-exec
> > ---
> > [INFO] Setting property: classpath.resource.loader.class =>
> > 'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'.
> > [INFO] Setting property: velocimacro.messages.on => 'false'.
> > [INFO] Setting property: resource.loader => 'classpath'.
> > [INFO] Setting property: resource.manager.logwhenfound => 'false'.
> > [INFO]
> > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
> > java-exec ---
> > [INFO] Using 'UTF-8' encoding to copy filtered resources.
> > [INFO] Copying 1 resource
> > [INFO] Copying 3 resources
> > [INFO]
> > [INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @
> java-exec
> > ---
> > [INFO] Changes detected - recompiling the module!
> > [INFO] Compiling 460 source files to
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/classes
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetStorageEngine.java:[20,21]
> > com.sun.corba.se.impl.interceptors.CodecFactoryImpl is internal
> proprietary
> > API and may be removed in a future release
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/bit/BitConnection.java:
> > Some input files use or override a deprecated API.
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/bit/BitConnection.java:
> > Recompile with -Xlint:deprecation for details.
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java:
> > Some input files use unchecked or unsafe operations.
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java:
> > Recompile with -Xlint:unchecked for details.
> > [INFO]
> > [INFO] --- maven-resources-plugin:2.6:testResources
> (default-testResources)
> > @ java-exec ---
> > [INFO] Using 'UTF-8' encoding to copy filtered resources.
> > [INFO] Copying 37 resources
> > [INFO] Copying 3 resources
> > [INFO]
> > [INFO] --- maven-compiler-plugin:3.0:testCompile (default-testCompile) @
> > java-exec ---
> > [INFO] Changes detected - recompiling the module!
> > [INFO] Compiling 127 source files to
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/test-classes
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[25,16]
> > sun.misc.Unsafe is internal proprietary API and may be removed in a
> future
> > release
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[53,17]
> > sun.misc.Unsafe is internal proprietary API and may be removed in a
> future
> > release
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[94,12]
> > sun.misc.Unsafe is internal proprietary API and may be removed in a
> future
> > release
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[95,14]
> > sun.misc.Unsafe is internal proprietary API and may be removed in a
> future
> > release
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[98,26]
> > sun.misc.Unsafe is internal proprietary API and may be removed in a
> future
> > release
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[100,27]
> > sun.misc.Unsafe is internal proprietary API and may be removed in a
> future
> > release
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/TestParquetPhysicalPlan.java:
> > Some input files use or override a deprecated API.
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/TestParquetPhysicalPlan.java:
> > Recompile with -Xlint:deprecation for details.
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java:
> > Some input files use unchecked or unsafe operations.
> > [WARNING]
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java:
> > Recompile with -Xlint:unchecked for details.
> > [INFO]
> > [INFO] --- maven-surefire-plugin:2.15:test (default-test) @ java-exec ---
> > [INFO] Surefire report directory:
> >
> >
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/surefire-reports
> >
> > -------------------------------------------------------
> > T E S T S
> > -------------------------------------------------------
> > Running org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
> > 09:23:52.569 [main] DEBUG org.apache.hadoop.security.Groups - Creating
> new
> > Groups object
> > 09:23:52.662 [main] DEBUG org.apache.hadoop.security.Groups - Group
> mapping
> > impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping;
> > cacheTimeout=300000
> > 09:23:52.841 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop
> > login
> > 09:23:52.842 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop
> > login commit
> > 09:23:52.863 [main] DEBUG o.a.h.security.UserGroupInformation - using
> local
> > user:UnixPrincipal: tanujit
> > 09:23:52.867 [main] DEBUG o.a.h.security.UserGroupInformation - UGI
> > loginUser:tanujit
> > 09:23:52.921 [main] DEBUG org.apache.hadoop.fs.FileSystem - Creating
> > filesystem for file:///
> > 09:23:53.011 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - Trying to
> load
> > the custom-built native-hadoop library...
> > 09:23:53.033 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - Failed to
> load
> > native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in
> > java.library.path
> > 09:23:53.034 [main] DEBUG o.a.hadoop.util.NativeCodeLoader -
> >
> >
> java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > 09:23:53.034 [main] WARN o.a.hadoop.util.NativeCodeLoader - Unable to
> load
> > native-hadoop library for your platform... using builtin-java classes
> where
> > applicable
> > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> > SLF4J: Defaulting to no-operation (NOP) logger implementation
> > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
> further
> > details.
> > 09:23:55.443 [main] DEBUG org.reflections.Reflections - going to scan
> these
> > urls:
> >
> >
> jar:file:/home/tanujit/.m2/repository/org/apache/drill/common/1.0-SNAPSHOT/common-1.0-SNAPSHOT.jar!/
> >
> >
> file:/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/test-classes/
> >
> >
> file:/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/classes/
> >
> >
> jar:file:/home/tanujit/.m2/repository/org/apache/drill/common/1.0-SNAPSHOT/common-1.0-SNAPSHOT-tests.jar!/
> >
> > 09:23:56.000 [main] INFO org.reflections.Reflections - Reflections took
> > 556 ms to scan 4 urls, producing 669 keys and 1634 values
> > 09:23:56.028 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J
> as
> > the default logging framework
> > 09:23:56.037 [main] DEBUG i.n.c.MultithreadEventLoopGroup -
> > -Dio.netty.eventLoopThreads: 8
> > 09:23:56.074 [main] DEBUG i.n.util.internal.PlatformDependent - UID: 1000
> > 09:23:56.075 [main] DEBUG i.n.util.internal.PlatformDependent - Java
> > version: 7
> > 09:23:56.076 [main] DEBUG i.n.util.internal.PlatformDependent -
> > -Dio.netty.noUnsafe: false
> > 09:23:56.079 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> > java.nio.ByteBuffer.cleaner: available
> > 09:23:56.079 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> > java.nio.Buffer.address: available
> > 09:23:56.080 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> > sun.misc.Unsafe.theUnsafe: available
> > 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> > sun.misc.Unsafe.copyMemory: available
> > 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> > java.nio.Bits.unaligned: true
> > 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent -
> > sun.misc.Unsafe: available
> > 09:23:56.082 [main] DEBUG i.n.util.internal.PlatformDependent -
> > -Dio.netty.noJavassist: false
> > 09:23:56.145 [main] DEBUG i.n.util.internal.PlatformDependent -
> Javassist:
> > available
> > 09:23:56.145 [main] DEBUG i.n.util.internal.PlatformDependent -
> > -Dio.netty.noPreferDirect: false
> > 09:23:56.164 [main] DEBUG io.netty.channel.nio.NioEventLoop -
> > -Dio.netty.noKeySetOptimization: false
> > 09:23:56.164 [main] DEBUG io.netty.channel.nio.NioEventLoop -
> > -Dio.netty.selectorAutoRebuildThreshold: 512
> > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> > -Dio.netty.allocator.numHeapArenas: 4
> > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> > -Dio.netty.allocator.numDirectArenas: 4
> > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> > -Dio.netty.allocator.pageSize: 8192
> > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> > -Dio.netty.allocator.maxOrder: 11
> > 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> > -Dio.netty.allocator.chunkSize: 16777216
> > running bit
> > 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback interface: lo
> > 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback address:
> > /0:0:0:0:0:0:0:1%1 (primary)
> > 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback address: /
> > 127.0.0.1
> > 09:23:56.277 [main] DEBUG io.netty.util.NetUtil -
> > /proc/sys/net/core/somaxconn: 128
> > 09:23:56.288 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
> [id:
> > 0x036be036] REGISTERED
> > 09:23:56.290 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
> [id:
> > 0x036be036] BIND(0.0.0.0/0.0.0.0:31010)
> > 09:23:56.294 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
> [id:
> > 0x036be036, /0:0:0:0:0:0:0:0:31010] ACTIVE
> > 09:23:56.310 [BitServer-1] INFO i.n.handler.logging.LoggingHandler -
> [id:
> > 0xb8299b35] REGISTERED
> > 09:23:56.311 [BitServer-1] INFO i.n.handler.logging.LoggingHandler -
> [id:
> > 0xb8299b35] BIND(0.0.0.0/0.0.0.0:31011)
> > 09:23:56.311 [BitServer-1] INFO i.n.handler.logging.LoggingHandler -
> [id:
> > 0xb8299b35, /0:0:0:0:0:0:0:0:31011] ACTIVE
> > running client
> > connecting - legolas -- 31010
> > 09:23:56.545 [Client-1] DEBUG
> > i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
> > io.netty.util.internal.__matchers__.io.netty.buffer.ByteBufMatcher
> > 09:23:56.549 [Client-1] DEBUG
> > i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
> >
> >
> io.netty.util.internal.__matchers__.org.apache.drill.exec.rpc.OutboundRpcMessageMatcher
> > 09:23:56.550 [Client-1] DEBUG
> > i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
> >
> >
> io.netty.util.internal.__matchers__.org.apache.drill.exec.rpc.InboundRpcMessageMatcher
> > 09:23:56.554 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
> [id:
> > 0x036be036, /0:0:0:0:0:0:0:0:31010] RECEIVED: [id: 0xf046bccd, /
> > 192.168.0.103:32952 => /192.168.0.103:31010]
> > 09:23:56.568 [Client-1] DEBUG io.netty.util.ResourceLeakDetector -
> > -Dio.netty.noResourceLeakDetection: false
> > running the query
> > Screen [endpoint=address: "legolas"
> > user_port: 31010
> > bit_port: 31011
> > ,
> getChild()=org.apache.drill.exec.store.parquet.ParquetGroupScan@59c81460
> > ]
> > 09:24:00.420 [WorkManager-2] WARN io.netty.util.ResourceLeakDetector -
> > LEAK: ByteBuf was GC'd before being released correctly. The following
> > stack trace shows where the leaked object was created, rather than where
> > you failed to release it.
> > io.netty.util.ResourceLeakException:
> > io.netty.buffer.PooledUnsafeDirectByteBufL@659974ba
> > at
> >
> >
> io.netty.util.ResourceLeakDetector$DefaultResourceLeak.<init>(ResourceLeakDetector.java:174)
> > ~[netty-common-4.0.7.Final.jar:na]
> > at io.netty.util.ResourceLeakDetector.open(ResourceLeakDetector.java:116)
> > ~[netty-common-4.0.7.Final.jar:na]
> > at io.netty.buffer.PooledByteBufL.<init>(PooledByteBufL.java:45)
> > ~[netty-bufferl-4.0.7.Final.jar:na]
> > at
> >
> >
> io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:53)
> > ~[netty-bufferl-4.0.7.Final.jar:na]
> > at
> >
> >
> io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:32)
> > ~[netty-bufferl-4.0.7.Final.jar:na]
> > at
> >
> >
> io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:39)
> > ~[netty-bufferl-4.0.7.Final.jar:na]
> > at
> >
> >
> io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:36)
> > ~[netty-bufferl-4.0.7.Final.jar:na]
> > at io.netty.util.Recycler.get(Recycler.java:40)
> > ~[netty-common-4.0.7.Final.jar:na]
> > at
> >
> >
> io.netty.buffer.PooledUnsafeDirectByteBufL.newInstance(PooledUnsafeDirectByteBufL.java:44)
> > ~[netty-bufferl-4.0.7.Final.jar:na]
> > at io.netty.buffer.PoolArenaL$DirectArena.newByteBuf(PoolArenaL.java:397)
> > ~[netty-bufferl-4.0.7.Final.jar:na]
> > at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:93)
> > ~[netty-bufferl-4.0.7.Final.jar:na]
> > at
> >
> >
> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:235)
> > ~[netty-bufferl-4.0.7.Final.jar:na]
> > at
> >
> >
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132)
> > ~[netty-buffer-4.0.7.Final.jar:na]
> > at
> >
> >
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123)
> > ~[netty-buffer-4.0.7.Final.jar:na]
> > at
> >
> >
> org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35)
> > ~[classes/:na]
> > at org.apache.drill.exec.vector.BitVector.allocateNew(BitVector.java:54)
> > ~[classes/:na]
> > at
> >
> >
> org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:12)
> > ~[classes/:na]
> > at
> >
> org.apache.drill.exec.store.VectorHolder.allocateNew(VectorHolder.java:87)
> > ~[classes/:na]
> > at org.apache.drill.exec.store.VectorHolder.reset(VectorHolder.java:74)
> > ~[classes/:na]
> > at
> >
> >
> org.apache.drill.exec.store.parquet.ParquetRecordReader.resetBatch(ParquetRecordReader.java:276)
> > ~[classes/:na]
> > at
> >
> >
> org.apache.drill.exec.store.parquet.ParquetRecordReader.next(ParquetRecordReader.java:318)
> > ~[classes/:na]
> > at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:99)
> > ~[classes/:na]
> > at
> >
> >
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:77)
> > ~[classes/:na]
> > at org.apache.drill.exec.work.FragmentRunner.run(FragmentRunner.java:79)
> > ~[classes/:na]
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > ~[na:1.7.0_25]
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > ~[na:1.7.0_25]
> > at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
> > Took 10003 ms to run query
> > 09:24:07.614 [BitServer-1] INFO i.n.handler.logging.LoggingHandler -
> [id:
> > 0xb8299b35, /0:0:0:0:0:0:0:0:31011] UNREGISTERED
> > 09:24:07.614 [UserServer-1] INFO i.n.handler.logging.LoggingHandler -
> [id:
> > 0x036be036, /0:0:0:0:0:0:0:0:31010] UNREGISTERED
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.329
> sec
> > - in org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
> > 09:24:07.621 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Starting
> > clear of FileSystem cache with 1 elements.
> > 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Removing
> > filesystem for file:///
> > 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Removing
> > filesystem for file:///
> > 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Done
> > clearing cache
> >
> > Results :
> >
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> >
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] BUILD SUCCESS
> > [INFO]
> > ------------------------------------------------------------------------
> > [INFO] Total time: 55.501s
> > [INFO] Finished at: Tue Aug 27 09:24:07 IST 2013
> > [INFO] Final Memory: 36M/429M
> > [INFO]
> > ------------------------------------------------------------------------
> >
> >
> > --
> > Regards,
> > Tanujit
> >
>
Re: Error with ParquetRecordReaderTest
Posted by Steven Phillips <sp...@maprtech.com>.
Maybe I'm just not looking closely enough, but I don't see the problem. It
looks like the test passed without issue.
On Mon, Aug 26, 2013 at 9:06 PM, Tanujit Ghosh <ta...@gmail.com>wrote:
> Hi,
>
> I'm facing this issue when i try to run the test in
>
> org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
>
> [tanujit@legolas java-exec]$ mvn
> -Dtest=ParquetRecordReaderTest#testMultipleRowGroupsAndReads test
> [INFO] Scanning for projects...
> [WARNING]
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.drill.exec:java-exec:jar:1.0-SNAPSHOT
> [WARNING] Detected profiles.xml alongside
> org.apache.drill.exec:java-exec:1.0-SNAPSHOT, this file is no longer
> supported and was ignored, please use the settings.xml instead
> [WARNING]
> [WARNING] It is highly recommended to fix these problems because they
> threaten the stability of your build.
> [WARNING]
> [WARNING] For this reason, future Maven versions might no longer support
> building such malformed projects.
> [WARNING]
> [INFO]
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Building java-exec 1.0-SNAPSHOT
> [INFO]
> ------------------------------------------------------------------------
> Downloading:
>
> http://repository.apache.org/snapshots/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> http://conjars.org/repo/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/common/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> http://repository.apache.org/snapshots/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> http://conjars.org/repo/org/apache/drill/prototype-parent/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> http://repository.apache.org/snapshots/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> http://conjars.org/repo/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
> Downloading:
>
> https://oss.sonatype.org/content/repositories/snapshots/org/apache/drill/exec/exec-parent/1.0-SNAPSHOT/maven-metadata.xml
> [INFO]
> [INFO] --- maven-resources-plugin:2.6:copy-resources (copy-resources) @
> java-exec ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 370 resources
> [INFO]
> [INFO] --- maven-enforcer-plugin:1.2:enforce (no_commons_logging) @
> java-exec ---
> [INFO]
> [INFO] --- maven-antrun-plugin:1.6:run (generate-sources) @ java-exec ---
> [WARNING] Parameter tasks is deprecated, use target instead
> [INFO] Executing tasks
>
> main:
> [INFO] Executed tasks
> [INFO] Registering compile source root
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/generated-sources
> [INFO]
> [INFO] --- fmpp-maven-plugin:1.0:generate (generate-sources) @ java-exec
> ---
> Downloading:
>
> https://oss.sonatype.org/content/repositories/snapshots/org/freemarker/freemarker/maven-metadata.xml
> Downloading:
>
> https://oss.sonatype.org/content/repositories/snapshots/oro/oro/maven-metadata.xml
> Downloading:
>
> https://oss.sonatype.org/content/repositories/snapshots/org/beanshell/bsh/maven-metadata.xml
> Downloading:
>
> https://oss.sonatype.org/content/repositories/snapshots/xml-resolver/xml-resolver/maven-metadata.xml
> - Executing: ValueHolders.java
> log4j:WARN No appenders could be found for logger (freemarker.cache).
> log4j:WARN Please initialize the log4j system properly.
> - Executing: VariableLengthVectors.java
> - Executing: FixedValueVectors.java
> - Executing: TypeHelper.java
> - Executing: NullableValueVectors.java
> - Executing: RepeatedValueVectors.java
> [INFO] Done
> [INFO]
> [INFO] --- maven-remote-resources-plugin:1.1:process (default) @ java-exec
> ---
> [INFO] Setting property: classpath.resource.loader.class =>
> 'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'.
> [INFO] Setting property: velocimacro.messages.on => 'false'.
> [INFO] Setting property: resource.loader => 'classpath'.
> [INFO] Setting property: resource.manager.logwhenfound => 'false'.
> [INFO]
> [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
> java-exec ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 1 resource
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @ java-exec
> ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 460 source files to
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/classes
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetStorageEngine.java:[20,21]
> com.sun.corba.se.impl.interceptors.CodecFactoryImpl is internal proprietary
> API and may be removed in a future release
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/bit/BitConnection.java:
> Some input files use or override a deprecated API.
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/rpc/bit/BitConnection.java:
> Recompile with -Xlint:deprecation for details.
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java:
> Some input files use unchecked or unsafe operations.
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java:
> Recompile with -Xlint:unchecked for details.
> [INFO]
> [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources)
> @ java-exec ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 37 resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:3.0:testCompile (default-testCompile) @
> java-exec ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 127 source files to
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/test-classes
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[25,16]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[53,17]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[94,12]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[95,14]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[98,26]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestExecutionAbstractions.java:[100,27]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/TestParquetPhysicalPlan.java:
> Some input files use or override a deprecated API.
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/TestParquetPhysicalPlan.java:
> Recompile with -Xlint:deprecation for details.
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java:
> Some input files use unchecked or unsafe operations.
> [WARNING]
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFileGenerator.java:
> Recompile with -Xlint:unchecked for details.
> [INFO]
> [INFO] --- maven-surefire-plugin:2.15:test (default-test) @ java-exec ---
> [INFO] Surefire report directory:
>
> /backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/surefire-reports
>
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
> 09:23:52.569 [main] DEBUG org.apache.hadoop.security.Groups - Creating new
> Groups object
> 09:23:52.662 [main] DEBUG org.apache.hadoop.security.Groups - Group mapping
> impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping;
> cacheTimeout=300000
> 09:23:52.841 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop
> login
> 09:23:52.842 [main] DEBUG o.a.h.security.UserGroupInformation - hadoop
> login commit
> 09:23:52.863 [main] DEBUG o.a.h.security.UserGroupInformation - using local
> user:UnixPrincipal: tanujit
> 09:23:52.867 [main] DEBUG o.a.h.security.UserGroupInformation - UGI
> loginUser:tanujit
> 09:23:52.921 [main] DEBUG org.apache.hadoop.fs.FileSystem - Creating
> filesystem for file:///
> 09:23:53.011 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - Trying to load
> the custom-built native-hadoop library...
> 09:23:53.033 [main] DEBUG o.a.hadoop.util.NativeCodeLoader - Failed to load
> native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in
> java.library.path
> 09:23:53.034 [main] DEBUG o.a.hadoop.util.NativeCodeLoader -
>
> java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 09:23:53.034 [main] WARN o.a.hadoop.util.NativeCodeLoader - Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> 09:23:55.443 [main] DEBUG org.reflections.Reflections - going to scan these
> urls:
>
> jar:file:/home/tanujit/.m2/repository/org/apache/drill/common/1.0-SNAPSHOT/common-1.0-SNAPSHOT.jar!/
>
> file:/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/test-classes/
>
> file:/backup/java/wip/incubator-drill/sandbox/prototype/exec/java-exec/target/classes/
>
> jar:file:/home/tanujit/.m2/repository/org/apache/drill/common/1.0-SNAPSHOT/common-1.0-SNAPSHOT-tests.jar!/
>
> 09:23:56.000 [main] INFO org.reflections.Reflections - Reflections took
> 556 ms to scan 4 urls, producing 669 keys and 1634 values
> 09:23:56.028 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as
> the default logging framework
> 09:23:56.037 [main] DEBUG i.n.c.MultithreadEventLoopGroup -
> -Dio.netty.eventLoopThreads: 8
> 09:23:56.074 [main] DEBUG i.n.util.internal.PlatformDependent - UID: 1000
> 09:23:56.075 [main] DEBUG i.n.util.internal.PlatformDependent - Java
> version: 7
> 09:23:56.076 [main] DEBUG i.n.util.internal.PlatformDependent -
> -Dio.netty.noUnsafe: false
> 09:23:56.079 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> java.nio.ByteBuffer.cleaner: available
> 09:23:56.079 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> java.nio.Buffer.address: available
> 09:23:56.080 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> sun.misc.Unsafe.theUnsafe: available
> 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> sun.misc.Unsafe.copyMemory: available
> 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent0 -
> java.nio.Bits.unaligned: true
> 09:23:56.081 [main] DEBUG i.n.util.internal.PlatformDependent -
> sun.misc.Unsafe: available
> 09:23:56.082 [main] DEBUG i.n.util.internal.PlatformDependent -
> -Dio.netty.noJavassist: false
> 09:23:56.145 [main] DEBUG i.n.util.internal.PlatformDependent - Javassist:
> available
> 09:23:56.145 [main] DEBUG i.n.util.internal.PlatformDependent -
> -Dio.netty.noPreferDirect: false
> 09:23:56.164 [main] DEBUG io.netty.channel.nio.NioEventLoop -
> -Dio.netty.noKeySetOptimization: false
> 09:23:56.164 [main] DEBUG io.netty.channel.nio.NioEventLoop -
> -Dio.netty.selectorAutoRebuildThreshold: 512
> 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> -Dio.netty.allocator.numHeapArenas: 4
> 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> -Dio.netty.allocator.numDirectArenas: 4
> 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> -Dio.netty.allocator.pageSize: 8192
> 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> -Dio.netty.allocator.maxOrder: 11
> 09:23:56.200 [main] DEBUG i.n.buffer.PooledByteBufAllocatorL -
> -Dio.netty.allocator.chunkSize: 16777216
> running bit
> 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback interface: lo
> 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback address:
> /0:0:0:0:0:0:0:1%1 (primary)
> 09:23:56.277 [main] DEBUG io.netty.util.NetUtil - Loopback address: /
> 127.0.0.1
> 09:23:56.277 [main] DEBUG io.netty.util.NetUtil -
> /proc/sys/net/core/somaxconn: 128
> 09:23:56.288 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0x036be036] REGISTERED
> 09:23:56.290 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0x036be036] BIND(0.0.0.0/0.0.0.0:31010)
> 09:23:56.294 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0x036be036, /0:0:0:0:0:0:0:0:31010] ACTIVE
> 09:23:56.310 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0xb8299b35] REGISTERED
> 09:23:56.311 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0xb8299b35] BIND(0.0.0.0/0.0.0.0:31011)
> 09:23:56.311 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0xb8299b35, /0:0:0:0:0:0:0:0:31011] ACTIVE
> running client
> connecting - legolas -- 31010
> 09:23:56.545 [Client-1] DEBUG
> i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
> io.netty.util.internal.__matchers__.io.netty.buffer.ByteBufMatcher
> 09:23:56.549 [Client-1] DEBUG
> i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
>
> io.netty.util.internal.__matchers__.org.apache.drill.exec.rpc.OutboundRpcMessageMatcher
> 09:23:56.550 [Client-1] DEBUG
> i.n.u.i.JavassistTypeParameterMatcherGenerator - Generated:
>
> io.netty.util.internal.__matchers__.org.apache.drill.exec.rpc.InboundRpcMessageMatcher
> 09:23:56.554 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0x036be036, /0:0:0:0:0:0:0:0:31010] RECEIVED: [id: 0xf046bccd, /
> 192.168.0.103:32952 => /192.168.0.103:31010]
> 09:23:56.568 [Client-1] DEBUG io.netty.util.ResourceLeakDetector -
> -Dio.netty.noResourceLeakDetection: false
> running the query
> Screen [endpoint=address: "legolas"
> user_port: 31010
> bit_port: 31011
> , getChild()=org.apache.drill.exec.store.parquet.ParquetGroupScan@59c81460
> ]
> 09:24:00.420 [WorkManager-2] WARN io.netty.util.ResourceLeakDetector -
> LEAK: ByteBuf was GC'd before being released correctly. The following
> stack trace shows where the leaked object was created, rather than where
> you failed to release it.
> io.netty.util.ResourceLeakException:
> io.netty.buffer.PooledUnsafeDirectByteBufL@659974ba
> at
>
> io.netty.util.ResourceLeakDetector$DefaultResourceLeak.<init>(ResourceLeakDetector.java:174)
> ~[netty-common-4.0.7.Final.jar:na]
> at io.netty.util.ResourceLeakDetector.open(ResourceLeakDetector.java:116)
> ~[netty-common-4.0.7.Final.jar:na]
> at io.netty.buffer.PooledByteBufL.<init>(PooledByteBufL.java:45)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:53)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.PooledUnsafeDirectByteBufL.<init>(PooledUnsafeDirectByteBufL.java:32)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:39)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.PooledUnsafeDirectByteBufL$1.newObject(PooledUnsafeDirectByteBufL.java:36)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at io.netty.util.Recycler.get(Recycler.java:40)
> ~[netty-common-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.PooledUnsafeDirectByteBufL.newInstance(PooledUnsafeDirectByteBufL.java:44)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at io.netty.buffer.PoolArenaL$DirectArena.newByteBuf(PoolArenaL.java:397)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:93)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.PooledByteBufAllocatorL.newDirectBuffer(PooledByteBufAllocatorL.java:235)
> ~[netty-bufferl-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:132)
> ~[netty-buffer-4.0.7.Final.jar:na]
> at
>
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:123)
> ~[netty-buffer-4.0.7.Final.jar:na]
> at
>
> org.apache.drill.exec.memory.DirectBufferAllocator.buffer(DirectBufferAllocator.java:35)
> ~[classes/:na]
> at org.apache.drill.exec.vector.BitVector.allocateNew(BitVector.java:54)
> ~[classes/:na]
> at
>
> org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:12)
> ~[classes/:na]
> at
> org.apache.drill.exec.store.VectorHolder.allocateNew(VectorHolder.java:87)
> ~[classes/:na]
> at org.apache.drill.exec.store.VectorHolder.reset(VectorHolder.java:74)
> ~[classes/:na]
> at
>
> org.apache.drill.exec.store.parquet.ParquetRecordReader.resetBatch(ParquetRecordReader.java:276)
> ~[classes/:na]
> at
>
> org.apache.drill.exec.store.parquet.ParquetRecordReader.next(ParquetRecordReader.java:318)
> ~[classes/:na]
> at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:99)
> ~[classes/:na]
> at
>
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:77)
> ~[classes/:na]
> at org.apache.drill.exec.work.FragmentRunner.run(FragmentRunner.java:79)
> ~[classes/:na]
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_25]
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ~[na:1.7.0_25]
> at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
> Took 10003 ms to run query
> 09:24:07.614 [BitServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0xb8299b35, /0:0:0:0:0:0:0:0:31011] UNREGISTERED
> 09:24:07.614 [UserServer-1] INFO i.n.handler.logging.LoggingHandler - [id:
> 0x036be036, /0:0:0:0:0:0:0:0:31010] UNREGISTERED
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.329 sec
> - in org.apache.drill.exec.store.parquet.ParquetRecordReaderTest
> 09:24:07.621 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Starting
> clear of FileSystem cache with 1 elements.
> 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Removing
> filesystem for file:///
> 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Removing
> filesystem for file:///
> 09:24:07.623 [Thread-1] DEBUG org.apache.hadoop.fs.FileSystem - Done
> clearing cache
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 55.501s
> [INFO] Finished at: Tue Aug 27 09:24:07 IST 2013
> [INFO] Final Memory: 36M/429M
> [INFO]
> ------------------------------------------------------------------------
>
>
> --
> Regards,
> Tanujit
>