You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Tridib Samanta <tr...@live.com> on 2014/10/30 20:28:47 UTC
Aggregation sum() fails on parquet file
Hello,
I am running a query like: select sum(amount) from myhdfs.parquet.`claim.parquet`;
This fails with the following error at sqlline:
Query failed: Failure while running fragment. Schema is currently null. You must call buildSchema(SelectionVectorMode) before this container can return a schema. [00314ea1-342a-4322-ac00-f8307c554733]
Error: exception while executing query: Failure while trying to get next result batch. (state=,code=0)
Here is the stack trace from drillbits.log:
2014-10-30 15:12:05,865 [3fb04943-c0e5-4801-a907-2bc3462a2997:frag:2:10] WARN o.a.d.e.w.fragment.FragmentExecutor - Failure while closing out resources.
java.lang.IllegalStateException: Failure while trying to close child allocator: Child level allocators not closed. Fragment 2:10. Stack trace:
org.apache.drill.exec.ops.OperatorContext.<init>(OperatorContext.java:41)
org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:90)
org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:148)
org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:54)
org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan(AbstractPhysicalVisitor.java:113)
org.apache.drill.exec.store.parquet.ParquetRowGroupScan.accept(ParquetRowGroupScan.java:107)
org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashAggregate(AbstractPhysicalVisitor.java:93)
org.apache.drill.exec.physical.config.HashAggregate.accept(HashAggregate.java:62)
org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:59)
org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSender(AbstractPhysicalVisitor.java:98)
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashPartitionSender(AbstractPhysicalVisitor.java:141)
org.apache.drill.exec.physical.config.HashPartitionSender.accept(HashPartitionSender.java:66)
org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:87)
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:94)
org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
at org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:215) ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
at org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:271) ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
at org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:139) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:107) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_67]
Any help is greatly appreciated.
Thanks & Regards
Tridib
RE: Aggregation sum() fails on parquet file
Posted by Tridib Samanta <tr...@live.com>.
Nope. Here is exception. This only happens when I am running it against parquet file in HDFS.
org.apache.drill.common.exceptions.DrillRuntimeException: java.io.IOException: FAILED_TO_UNCOMPRESS(5)
at org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:427) ~[drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:188) [drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
at org.apache.drill.exec.physical.impl.ScanBatch.buildSchema(ScanBatch.java:122) [drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.buildSchema(IteratorValidatorBatchIterator.java:80) [drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
at org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.buildSchema(HashAggBatch.java:97) [drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.buildSchema(IteratorValidatorBatchIterator.java:80) [drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
at org.apache.drill.exec.physical.impl.partitionsender.PartitionSenderRootExec.buildSchema(PartitionSenderRootExec.java:115) [drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:103) [drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:249) [drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_67]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
Caused by: java.io.IOException: FAILED_TO_UNCOMPRESS(5)
at org.xerial.snappy.SnappyNative.throw_error(SnappyNative.java:78) ~[na:na]
at org.xerial.snappy.SnappyNative.rawUncompress(Native Method) ~[na:na]
at org.xerial.snappy.Snappy.uncompress(Snappy.java:467) ~[snappy-java-1.0.5-M3.jar:na]
at parquet.hadoop.codec.SnappyDecompressor$SnappyDirectDecompressor.decompress(SnappyDecompressor.java:160) ~[parquet-hadoop-1.5.1-drill-r4.jar:na]
at parquet.hadoop.CodecFactoryExposer.decompress(CodecFactoryExposer.java:88) ~[drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
at org.apache.drill.exec.store.parquet.columnreaders.PageReader.next(PageReader.java:212) ~[drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
at org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.readPage(ColumnReader.java:198) ~[drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
at org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:141) ~[drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
at org.apache.drill.exec.store.parquet.columnreaders.VarLenBinaryReader.readFields(VarLenBinaryReader.java:56) ~[drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
at org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:411) ~[drill-java-exec-0.7.0-incubating-SNAPSHOT-rebuffed.jar:0.7.0-incubating-SNAPSHOT]
... 11 common frames omitted
Thanks
Tridib
> From: danielrulez@gmail.com
> Subject: Re: Aggregation sum() fails on parquet file
> Date: Fri, 31 Oct 2014 09:30:03 +0200
> To: drill-user@incubator.apache.org
>
> And did it solve your problem?
>
> Daniel
>
> > On 31 באוק׳ 2014, at 09:25, Tridib Samanta <tr...@live.com> wrote:
> >
> > Thanks. I am able to build and run 0.7.0-SNAPSHOT.
> >
> >> From: altekrusejason@gmail.com
> >> Date: Thu, 30 Oct 2014 23:50:14 -0700
> >> Subject: Re: Aggregation sum() fails on parquet file
> >> To: drill-user@incubator.apache.org
> >>
> >> Here is the wiki page on building from source. I am not much of a maven
> >> expert, but the dev team uses 'mvn install' to run builds, not sure how it
> >> differs from mvn package.
> >>
> >> https://cwiki.apache.org/confluence/display/DRILL/Compiling+Drill+from+Source
> >>
> >> If you want to poke around at all in the code when you start building it
> >> there are a number of articles linked from the main wiki page about setting
> >> up the source in an IDE and contributing back patches.
> >>
> >> https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+Wiki
> >>
> >> -Jason Altekruse
> >>
> >> On Thu, Oct 30, 2014 at 11:27 PM, Tridib Samanta <tr...@live.com>
> >> wrote:
> >>
> >>> Nope. I am running 0.5. I tried to build from latest master with 'mvn
> >>> package'. It failed with error .git not found. I wish there was a document
> >>> how to build from source.
> >>>
> >>>> Date: Thu, 30 Oct 2014 23:14:17 -0700
> >>>> Subject: Re: Aggregation sum() fails on parquet file
> >>>> From: tnachen@gmail.com
> >>>> To: drill-user@incubator.apache.org
> >>>>
> >>>> Hi there,
> >>>>
> >>>> Are you running the latest master?
> >>>>
> >>>> Tim
> >>>>
> >>>> On Thu, Oct 30, 2014 at 10:57 PM, Tridib Samanta
> >>>> <tr...@live.com> wrote:
> >>>>> No clue. :(. Now I am struggling to run distinct on parquet file.
> >>>>>
> >>>>>> From: danielrulez@gmail.com
> >>>>>> Subject: Re: Aggregation sum() fails on parquet file
> >>>>>> Date: Fri, 31 Oct 2014 07:14:43 +0200
> >>>>>> To: drill-user@incubator.apache.org
> >>>>>>
> >>>>>> Hi,
> >>>>>> Any clue on that? I'm facing a similar problem
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Daniel
> >>>>>>
> >>>>>>> On 30 באוק׳ 2014, at 21:28, Tridib Samanta <tr...@live.com>
> >>> wrote:
> >>>>>>>
> >>>>>>> Hello,
> >>>>>>> I am running a query like: select sum(amount) from
> >>> myhdfs.parquet.`claim.parquet`;
> >>>>>>> This fails with the following error at sqlline:
> >>>>>>>
> >>>>>>> Query failed: Failure while running fragment. Schema is currently
> >>> null. You must call buildSchema(SelectionVectorMode) before this container
> >>> can return a schema. [00314ea1-342a-4322-ac00-f8307c554733]
> >>>>>>> Error: exception while executing query: Failure while trying to get
> >>> next result batch. (state=,code=0)
> >>>>>>>
> >>>>>>>
> >>>>>>> Here is the stack trace from drillbits.log:
> >>>>>>>
> >>>>>>> 2014-10-30 15:12:05,865
> >>> [3fb04943-c0e5-4801-a907-2bc3462a2997:frag:2:10] WARN
> >>> o.a.d.e.w.fragment.FragmentExecutor - Failure while closing out resources.
> >>>>>>> java.lang.IllegalStateException: Failure while trying to close
> >>> child allocator: Child level allocators not closed. Fragment 2:10. Stack
> >>> trace:
> >>> org.apache.drill.exec.ops.OperatorContext.<init>(OperatorContext.java:41)
> >>> org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:90)
> >>> org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:148)
> >>> org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:54)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> >>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan(AbstractPhysicalVisitor.java:113)
> >>> org.apache.drill.exec.store.parquet.ParquetRowGroupScan.accept(ParquetRowGroupScan.java:107)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> >>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
> >>> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> >>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashAggregate(AbstractPhysicalVisitor.java:93)
> >>> org.apache.drill.exec.physical.config.HashAggregate.accept(HashAggregate.java:62)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> >>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
> >>> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:59)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> >>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSender(AbstractPhysicalVisitor.java:98)
> >>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashPartitionSender(AbstractPhysicalVisitor.java:141)
> >>> org.apache.drill.exec.physical.config.HashPartitionSender.accept(HashPartitionSender.java:66)
> >>> org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:87)
> >>> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:94)
> >>> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
> >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >>>>>>> java.lang.Thread.run(Thread.java:745)
> >>>>>>> at
> >>> org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:215)
> >>> ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> >>>>>>> at
> >>> org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:271)
> >>> ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> >>>>>>> at
> >>> org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:139)
> >>> [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> >>>>>>> at
> >>> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:107)
> >>> [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> >>>>>>> at
> >>> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
> >>> [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> >>>>>>> at
> >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >>> [na:1.7.0_67]
> >>>>>>> at
> >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >>> [na:1.7.0_67]
> >>>>>>>
> >>>>>>> Any help is greatly appreciated.
> >>>>>>>
> >>>>>>> Thanks & Regards
> >>>>>>> Tridib
> >
Re: Aggregation sum() fails on parquet file
Posted by Daniel Haviv <da...@gmail.com>.
And did it solve your problem?
Daniel
> On 31 באוק׳ 2014, at 09:25, Tridib Samanta <tr...@live.com> wrote:
>
> Thanks. I am able to build and run 0.7.0-SNAPSHOT.
>
>> From: altekrusejason@gmail.com
>> Date: Thu, 30 Oct 2014 23:50:14 -0700
>> Subject: Re: Aggregation sum() fails on parquet file
>> To: drill-user@incubator.apache.org
>>
>> Here is the wiki page on building from source. I am not much of a maven
>> expert, but the dev team uses 'mvn install' to run builds, not sure how it
>> differs from mvn package.
>>
>> https://cwiki.apache.org/confluence/display/DRILL/Compiling+Drill+from+Source
>>
>> If you want to poke around at all in the code when you start building it
>> there are a number of articles linked from the main wiki page about setting
>> up the source in an IDE and contributing back patches.
>>
>> https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+Wiki
>>
>> -Jason Altekruse
>>
>> On Thu, Oct 30, 2014 at 11:27 PM, Tridib Samanta <tr...@live.com>
>> wrote:
>>
>>> Nope. I am running 0.5. I tried to build from latest master with 'mvn
>>> package'. It failed with error .git not found. I wish there was a document
>>> how to build from source.
>>>
>>>> Date: Thu, 30 Oct 2014 23:14:17 -0700
>>>> Subject: Re: Aggregation sum() fails on parquet file
>>>> From: tnachen@gmail.com
>>>> To: drill-user@incubator.apache.org
>>>>
>>>> Hi there,
>>>>
>>>> Are you running the latest master?
>>>>
>>>> Tim
>>>>
>>>> On Thu, Oct 30, 2014 at 10:57 PM, Tridib Samanta
>>>> <tr...@live.com> wrote:
>>>>> No clue. :(. Now I am struggling to run distinct on parquet file.
>>>>>
>>>>>> From: danielrulez@gmail.com
>>>>>> Subject: Re: Aggregation sum() fails on parquet file
>>>>>> Date: Fri, 31 Oct 2014 07:14:43 +0200
>>>>>> To: drill-user@incubator.apache.org
>>>>>>
>>>>>> Hi,
>>>>>> Any clue on that? I'm facing a similar problem
>>>>>>
>>>>>> Thanks,
>>>>>> Daniel
>>>>>>
>>>>>>> On 30 באוק׳ 2014, at 21:28, Tridib Samanta <tr...@live.com>
>>> wrote:
>>>>>>>
>>>>>>> Hello,
>>>>>>> I am running a query like: select sum(amount) from
>>> myhdfs.parquet.`claim.parquet`;
>>>>>>> This fails with the following error at sqlline:
>>>>>>>
>>>>>>> Query failed: Failure while running fragment. Schema is currently
>>> null. You must call buildSchema(SelectionVectorMode) before this container
>>> can return a schema. [00314ea1-342a-4322-ac00-f8307c554733]
>>>>>>> Error: exception while executing query: Failure while trying to get
>>> next result batch. (state=,code=0)
>>>>>>>
>>>>>>>
>>>>>>> Here is the stack trace from drillbits.log:
>>>>>>>
>>>>>>> 2014-10-30 15:12:05,865
>>> [3fb04943-c0e5-4801-a907-2bc3462a2997:frag:2:10] WARN
>>> o.a.d.e.w.fragment.FragmentExecutor - Failure while closing out resources.
>>>>>>> java.lang.IllegalStateException: Failure while trying to close
>>> child allocator: Child level allocators not closed. Fragment 2:10. Stack
>>> trace:
>>> org.apache.drill.exec.ops.OperatorContext.<init>(OperatorContext.java:41)
>>> org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:90)
>>> org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:148)
>>> org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:54)
>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan(AbstractPhysicalVisitor.java:113)
>>> org.apache.drill.exec.store.parquet.ParquetRowGroupScan.accept(ParquetRowGroupScan.java:107)
>>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
>>> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
>>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashAggregate(AbstractPhysicalVisitor.java:93)
>>> org.apache.drill.exec.physical.config.HashAggregate.accept(HashAggregate.java:62)
>>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
>>> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
>>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:59)
>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSender(AbstractPhysicalVisitor.java:98)
>>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashPartitionSender(AbstractPhysicalVisitor.java:141)
>>> org.apache.drill.exec.physical.config.HashPartitionSender.accept(HashPartitionSender.java:66)
>>> org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:87)
>>> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:94)
>>> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>>>> java.lang.Thread.run(Thread.java:745)
>>>>>>> at
>>> org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:215)
>>> ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>>>>>>> at
>>> org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:271)
>>> ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>>>>>>> at
>>> org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:139)
>>> [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>>>>>>> at
>>> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:107)
>>> [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>>>>>>> at
>>> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
>>> [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>>>>>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> [na:1.7.0_67]
>>>>>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> [na:1.7.0_67]
>>>>>>>
>>>>>>> Any help is greatly appreciated.
>>>>>>>
>>>>>>> Thanks & Regards
>>>>>>> Tridib
>
RE: Aggregation sum() fails on parquet file
Posted by Tridib Samanta <tr...@live.com>.
Thanks. I am able to build and run 0.7.0-SNAPSHOT.
> From: altekrusejason@gmail.com
> Date: Thu, 30 Oct 2014 23:50:14 -0700
> Subject: Re: Aggregation sum() fails on parquet file
> To: drill-user@incubator.apache.org
>
> Here is the wiki page on building from source. I am not much of a maven
> expert, but the dev team uses 'mvn install' to run builds, not sure how it
> differs from mvn package.
>
> https://cwiki.apache.org/confluence/display/DRILL/Compiling+Drill+from+Source
>
> If you want to poke around at all in the code when you start building it
> there are a number of articles linked from the main wiki page about setting
> up the source in an IDE and contributing back patches.
>
> https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+Wiki
>
> -Jason Altekruse
>
> On Thu, Oct 30, 2014 at 11:27 PM, Tridib Samanta <tr...@live.com>
> wrote:
>
> > Nope. I am running 0.5. I tried to build from latest master with 'mvn
> > package'. It failed with error .git not found. I wish there was a document
> > how to build from source.
> >
> > > Date: Thu, 30 Oct 2014 23:14:17 -0700
> > > Subject: Re: Aggregation sum() fails on parquet file
> > > From: tnachen@gmail.com
> > > To: drill-user@incubator.apache.org
> > >
> > > Hi there,
> > >
> > > Are you running the latest master?
> > >
> > > Tim
> > >
> > > On Thu, Oct 30, 2014 at 10:57 PM, Tridib Samanta
> > > <tr...@live.com> wrote:
> > > > No clue. :(. Now I am struggling to run distinct on parquet file.
> > > >
> > > >> From: danielrulez@gmail.com
> > > >> Subject: Re: Aggregation sum() fails on parquet file
> > > >> Date: Fri, 31 Oct 2014 07:14:43 +0200
> > > >> To: drill-user@incubator.apache.org
> > > >>
> > > >> Hi,
> > > >> Any clue on that? I'm facing a similar problem
> > > >>
> > > >> Thanks,
> > > >> Daniel
> > > >>
> > > >> > On 30 באוק׳ 2014, at 21:28, Tridib Samanta <tr...@live.com>
> > wrote:
> > > >> >
> > > >> > Hello,
> > > >> > I am running a query like: select sum(amount) from
> > myhdfs.parquet.`claim.parquet`;
> > > >> > This fails with the following error at sqlline:
> > > >> >
> > > >> > Query failed: Failure while running fragment. Schema is currently
> > null. You must call buildSchema(SelectionVectorMode) before this container
> > can return a schema. [00314ea1-342a-4322-ac00-f8307c554733]
> > > >> > Error: exception while executing query: Failure while trying to get
> > next result batch. (state=,code=0)
> > > >> >
> > > >> >
> > > >> > Here is the stack trace from drillbits.log:
> > > >> >
> > > >> > 2014-10-30 15:12:05,865
> > [3fb04943-c0e5-4801-a907-2bc3462a2997:frag:2:10] WARN
> > o.a.d.e.w.fragment.FragmentExecutor - Failure while closing out resources.
> > > >> > java.lang.IllegalStateException: Failure while trying to close
> > child allocator: Child level allocators not closed. Fragment 2:10. Stack
> > trace:
> > > >> >
> > org.apache.drill.exec.ops.OperatorContext.<init>(OperatorContext.java:41)
> > > >> >
> > org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:90)
> > > >> >
> > org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:148)
> > > >> >
> > org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:54)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > > >> >
> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan(AbstractPhysicalVisitor.java:113)
> > > >> >
> > org.apache.drill.exec.store.parquet.ParquetRowGroupScan.accept(ParquetRowGroupScan.java:107)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > > >> >
> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
> > > >> >
> > org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > > >> >
> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashAggregate(AbstractPhysicalVisitor.java:93)
> > > >> >
> > org.apache.drill.exec.physical.config.HashAggregate.accept(HashAggregate.java:62)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > > >> >
> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
> > > >> >
> > org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:59)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > > >> >
> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSender(AbstractPhysicalVisitor.java:98)
> > > >> >
> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashPartitionSender(AbstractPhysicalVisitor.java:141)
> > > >> >
> > org.apache.drill.exec.physical.config.HashPartitionSender.accept(HashPartitionSender.java:66)
> > > >> >
> > org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:87)
> > > >> >
> > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:94)
> > > >> >
> > org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
> > > >> >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > > >> >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > > >> > java.lang.Thread.run(Thread.java:745)
> > > >> > at
> > org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:215)
> > ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > > >> > at
> > org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:271)
> > ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > > >> > at
> > org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:139)
> > [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > > >> > at
> > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:107)
> > [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > > >> > at
> > org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
> > [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > > >> > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > [na:1.7.0_67]
> > > >> > at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > [na:1.7.0_67]
> > > >> >
> > > >> > Any help is greatly appreciated.
> > > >> >
> > > >> > Thanks & Regards
> > > >> > Tridib
> > > >> >
> > > >
> >
> >
Re: Aggregation sum() fails on parquet file
Posted by Jason Altekruse <al...@gmail.com>.
Here is the wiki page on building from source. I am not much of a maven
expert, but the dev team uses 'mvn install' to run builds, not sure how it
differs from mvn package.
https://cwiki.apache.org/confluence/display/DRILL/Compiling+Drill+from+Source
If you want to poke around at all in the code when you start building it
there are a number of articles linked from the main wiki page about setting
up the source in an IDE and contributing back patches.
https://cwiki.apache.org/confluence/display/DRILL/Apache+Drill+Wiki
-Jason Altekruse
On Thu, Oct 30, 2014 at 11:27 PM, Tridib Samanta <tr...@live.com>
wrote:
> Nope. I am running 0.5. I tried to build from latest master with 'mvn
> package'. It failed with error .git not found. I wish there was a document
> how to build from source.
>
> > Date: Thu, 30 Oct 2014 23:14:17 -0700
> > Subject: Re: Aggregation sum() fails on parquet file
> > From: tnachen@gmail.com
> > To: drill-user@incubator.apache.org
> >
> > Hi there,
> >
> > Are you running the latest master?
> >
> > Tim
> >
> > On Thu, Oct 30, 2014 at 10:57 PM, Tridib Samanta
> > <tr...@live.com> wrote:
> > > No clue. :(. Now I am struggling to run distinct on parquet file.
> > >
> > >> From: danielrulez@gmail.com
> > >> Subject: Re: Aggregation sum() fails on parquet file
> > >> Date: Fri, 31 Oct 2014 07:14:43 +0200
> > >> To: drill-user@incubator.apache.org
> > >>
> > >> Hi,
> > >> Any clue on that? I'm facing a similar problem
> > >>
> > >> Thanks,
> > >> Daniel
> > >>
> > >> > On 30 באוק׳ 2014, at 21:28, Tridib Samanta <tr...@live.com>
> wrote:
> > >> >
> > >> > Hello,
> > >> > I am running a query like: select sum(amount) from
> myhdfs.parquet.`claim.parquet`;
> > >> > This fails with the following error at sqlline:
> > >> >
> > >> > Query failed: Failure while running fragment. Schema is currently
> null. You must call buildSchema(SelectionVectorMode) before this container
> can return a schema. [00314ea1-342a-4322-ac00-f8307c554733]
> > >> > Error: exception while executing query: Failure while trying to get
> next result batch. (state=,code=0)
> > >> >
> > >> >
> > >> > Here is the stack trace from drillbits.log:
> > >> >
> > >> > 2014-10-30 15:12:05,865
> [3fb04943-c0e5-4801-a907-2bc3462a2997:frag:2:10] WARN
> o.a.d.e.w.fragment.FragmentExecutor - Failure while closing out resources.
> > >> > java.lang.IllegalStateException: Failure while trying to close
> child allocator: Child level allocators not closed. Fragment 2:10. Stack
> trace:
> > >> >
> org.apache.drill.exec.ops.OperatorContext.<init>(OperatorContext.java:41)
> > >> >
> org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:90)
> > >> >
> org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:148)
> > >> >
> org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:54)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > >> >
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan(AbstractPhysicalVisitor.java:113)
> > >> >
> org.apache.drill.exec.store.parquet.ParquetRowGroupScan.accept(ParquetRowGroupScan.java:107)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > >> >
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
> > >> >
> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > >> >
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashAggregate(AbstractPhysicalVisitor.java:93)
> > >> >
> org.apache.drill.exec.physical.config.HashAggregate.accept(HashAggregate.java:62)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > >> >
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
> > >> >
> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:59)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > >> >
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSender(AbstractPhysicalVisitor.java:98)
> > >> >
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashPartitionSender(AbstractPhysicalVisitor.java:141)
> > >> >
> org.apache.drill.exec.physical.config.HashPartitionSender.accept(HashPartitionSender.java:66)
> > >> >
> org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:87)
> > >> >
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:94)
> > >> >
> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
> > >> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > >> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > >> > java.lang.Thread.run(Thread.java:745)
> > >> > at
> org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:215)
> ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > >> > at
> org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:271)
> ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > >> > at
> org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:139)
> [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > >> > at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:107)
> [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > >> > at
> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
> [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > >> > at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_67]
> > >> > at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_67]
> > >> >
> > >> > Any help is greatly appreciated.
> > >> >
> > >> > Thanks & Regards
> > >> > Tridib
> > >> >
> > >
>
>
RE: Aggregation sum() fails on parquet file
Posted by Tridib Samanta <tr...@live.com>.
Nope. I am running 0.5. I tried to build from latest master with 'mvn package'. It failed with error .git not found. I wish there was a document how to build from source.
> Date: Thu, 30 Oct 2014 23:14:17 -0700
> Subject: Re: Aggregation sum() fails on parquet file
> From: tnachen@gmail.com
> To: drill-user@incubator.apache.org
>
> Hi there,
>
> Are you running the latest master?
>
> Tim
>
> On Thu, Oct 30, 2014 at 10:57 PM, Tridib Samanta
> <tr...@live.com> wrote:
> > No clue. :(. Now I am struggling to run distinct on parquet file.
> >
> >> From: danielrulez@gmail.com
> >> Subject: Re: Aggregation sum() fails on parquet file
> >> Date: Fri, 31 Oct 2014 07:14:43 +0200
> >> To: drill-user@incubator.apache.org
> >>
> >> Hi,
> >> Any clue on that? I'm facing a similar problem
> >>
> >> Thanks,
> >> Daniel
> >>
> >> > On 30 באוק׳ 2014, at 21:28, Tridib Samanta <tr...@live.com> wrote:
> >> >
> >> > Hello,
> >> > I am running a query like: select sum(amount) from myhdfs.parquet.`claim.parquet`;
> >> > This fails with the following error at sqlline:
> >> >
> >> > Query failed: Failure while running fragment. Schema is currently null. You must call buildSchema(SelectionVectorMode) before this container can return a schema. [00314ea1-342a-4322-ac00-f8307c554733]
> >> > Error: exception while executing query: Failure while trying to get next result batch. (state=,code=0)
> >> >
> >> >
> >> > Here is the stack trace from drillbits.log:
> >> >
> >> > 2014-10-30 15:12:05,865 [3fb04943-c0e5-4801-a907-2bc3462a2997:frag:2:10] WARN o.a.d.e.w.fragment.FragmentExecutor - Failure while closing out resources.
> >> > java.lang.IllegalStateException: Failure while trying to close child allocator: Child level allocators not closed. Fragment 2:10. Stack trace:
> >> > org.apache.drill.exec.ops.OperatorContext.<init>(OperatorContext.java:41)
> >> > org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:90)
> >> > org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:148)
> >> > org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:54)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> >> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan(AbstractPhysicalVisitor.java:113)
> >> > org.apache.drill.exec.store.parquet.ParquetRowGroupScan.accept(ParquetRowGroupScan.java:107)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> >> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
> >> > org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> >> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashAggregate(AbstractPhysicalVisitor.java:93)
> >> > org.apache.drill.exec.physical.config.HashAggregate.accept(HashAggregate.java:62)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> >> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
> >> > org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:59)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> >> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSender(AbstractPhysicalVisitor.java:98)
> >> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashPartitionSender(AbstractPhysicalVisitor.java:141)
> >> > org.apache.drill.exec.physical.config.HashPartitionSender.accept(HashPartitionSender.java:66)
> >> > org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:87)
> >> > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:94)
> >> > org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
> >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >> > java.lang.Thread.run(Thread.java:745)
> >> > at org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:215) ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> >> > at org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:271) ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> >> > at org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:139) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> >> > at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:107) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> >> > at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> >> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_67]
> >> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_67]
> >> >
> >> > Any help is greatly appreciated.
> >> >
> >> > Thanks & Regards
> >> > Tridib
> >> >
> >
Re: Aggregation sum() fails on parquet file
Posted by Daniel Haviv <da...@gmail.com>.
I'm using 0.5
Daniel
> On 31 באוק׳ 2014, at 08:14, Timothy Chen <tn...@gmail.com> wrote:
>
> Hi there,
>
> Are you running the latest master?
>
> Tim
>
> On Thu, Oct 30, 2014 at 10:57 PM, Tridib Samanta
> <tr...@live.com> wrote:
>> No clue. :(. Now I am struggling to run distinct on parquet file.
>>
>>> From: danielrulez@gmail.com
>>> Subject: Re: Aggregation sum() fails on parquet file
>>> Date: Fri, 31 Oct 2014 07:14:43 +0200
>>> To: drill-user@incubator.apache.org
>>>
>>> Hi,
>>> Any clue on that? I'm facing a similar problem
>>>
>>> Thanks,
>>> Daniel
>>>
>>>> On 30 באוק׳ 2014, at 21:28, Tridib Samanta <tr...@live.com> wrote:
>>>>
>>>> Hello,
>>>> I am running a query like: select sum(amount) from myhdfs.parquet.`claim.parquet`;
>>>> This fails with the following error at sqlline:
>>>>
>>>> Query failed: Failure while running fragment. Schema is currently null. You must call buildSchema(SelectionVectorMode) before this container can return a schema. [00314ea1-342a-4322-ac00-f8307c554733]
>>>> Error: exception while executing query: Failure while trying to get next result batch. (state=,code=0)
>>>>
>>>>
>>>> Here is the stack trace from drillbits.log:
>>>>
>>>> 2014-10-30 15:12:05,865 [3fb04943-c0e5-4801-a907-2bc3462a2997:frag:2:10] WARN o.a.d.e.w.fragment.FragmentExecutor - Failure while closing out resources.
>>>> java.lang.IllegalStateException: Failure while trying to close child allocator: Child level allocators not closed. Fragment 2:10. Stack trace:
>>>> org.apache.drill.exec.ops.OperatorContext.<init>(OperatorContext.java:41)
>>>> org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:90)
>>>> org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:148)
>>>> org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:54)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>>>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan(AbstractPhysicalVisitor.java:113)
>>>> org.apache.drill.exec.store.parquet.ParquetRowGroupScan.accept(ParquetRowGroupScan.java:107)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>>>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
>>>> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>>>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashAggregate(AbstractPhysicalVisitor.java:93)
>>>> org.apache.drill.exec.physical.config.HashAggregate.accept(HashAggregate.java:62)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>>>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
>>>> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:59)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>>>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSender(AbstractPhysicalVisitor.java:98)
>>>> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashPartitionSender(AbstractPhysicalVisitor.java:141)
>>>> org.apache.drill.exec.physical.config.HashPartitionSender.accept(HashPartitionSender.java:66)
>>>> org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:87)
>>>> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:94)
>>>> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>> java.lang.Thread.run(Thread.java:745)
>>>> at org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:215) ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>>>> at org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:271) ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>>>> at org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:139) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>>>> at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:107) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>>>> at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_67]
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_67]
>>>>
>>>> Any help is greatly appreciated.
>>>>
>>>> Thanks & Regards
>>>> Tridib
>>
Re: Aggregation sum() fails on parquet file
Posted by Timothy Chen <tn...@gmail.com>.
Hi there,
Are you running the latest master?
Tim
On Thu, Oct 30, 2014 at 10:57 PM, Tridib Samanta
<tr...@live.com> wrote:
> No clue. :(. Now I am struggling to run distinct on parquet file.
>
>> From: danielrulez@gmail.com
>> Subject: Re: Aggregation sum() fails on parquet file
>> Date: Fri, 31 Oct 2014 07:14:43 +0200
>> To: drill-user@incubator.apache.org
>>
>> Hi,
>> Any clue on that? I'm facing a similar problem
>>
>> Thanks,
>> Daniel
>>
>> > On 30 באוק׳ 2014, at 21:28, Tridib Samanta <tr...@live.com> wrote:
>> >
>> > Hello,
>> > I am running a query like: select sum(amount) from myhdfs.parquet.`claim.parquet`;
>> > This fails with the following error at sqlline:
>> >
>> > Query failed: Failure while running fragment. Schema is currently null. You must call buildSchema(SelectionVectorMode) before this container can return a schema. [00314ea1-342a-4322-ac00-f8307c554733]
>> > Error: exception while executing query: Failure while trying to get next result batch. (state=,code=0)
>> >
>> >
>> > Here is the stack trace from drillbits.log:
>> >
>> > 2014-10-30 15:12:05,865 [3fb04943-c0e5-4801-a907-2bc3462a2997:frag:2:10] WARN o.a.d.e.w.fragment.FragmentExecutor - Failure while closing out resources.
>> > java.lang.IllegalStateException: Failure while trying to close child allocator: Child level allocators not closed. Fragment 2:10. Stack trace:
>> > org.apache.drill.exec.ops.OperatorContext.<init>(OperatorContext.java:41)
>> > org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:90)
>> > org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:148)
>> > org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:54)
>> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
>> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan(AbstractPhysicalVisitor.java:113)
>> > org.apache.drill.exec.store.parquet.ParquetRowGroupScan.accept(ParquetRowGroupScan.java:107)
>> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
>> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
>> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
>> > org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
>> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
>> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
>> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashAggregate(AbstractPhysicalVisitor.java:93)
>> > org.apache.drill.exec.physical.config.HashAggregate.accept(HashAggregate.java:62)
>> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
>> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
>> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
>> > org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
>> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
>> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:59)
>> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
>> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSender(AbstractPhysicalVisitor.java:98)
>> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashPartitionSender(AbstractPhysicalVisitor.java:141)
>> > org.apache.drill.exec.physical.config.HashPartitionSender.accept(HashPartitionSender.java:66)
>> > org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:87)
>> > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:94)
>> > org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> > java.lang.Thread.run(Thread.java:745)
>> > at org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:215) ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>> > at org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:271) ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>> > at org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:139) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>> > at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:107) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>> > at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
>> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_67]
>> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_67]
>> >
>> > Any help is greatly appreciated.
>> >
>> > Thanks & Regards
>> > Tridib
>> >
>
RE: Aggregation sum() fails on parquet file
Posted by Tridib Samanta <tr...@live.com>.
No clue. :(. Now I am struggling to run distinct on parquet file.
> From: danielrulez@gmail.com
> Subject: Re: Aggregation sum() fails on parquet file
> Date: Fri, 31 Oct 2014 07:14:43 +0200
> To: drill-user@incubator.apache.org
>
> Hi,
> Any clue on that? I'm facing a similar problem
>
> Thanks,
> Daniel
>
> > On 30 באוק׳ 2014, at 21:28, Tridib Samanta <tr...@live.com> wrote:
> >
> > Hello,
> > I am running a query like: select sum(amount) from myhdfs.parquet.`claim.parquet`;
> > This fails with the following error at sqlline:
> >
> > Query failed: Failure while running fragment. Schema is currently null. You must call buildSchema(SelectionVectorMode) before this container can return a schema. [00314ea1-342a-4322-ac00-f8307c554733]
> > Error: exception while executing query: Failure while trying to get next result batch. (state=,code=0)
> >
> >
> > Here is the stack trace from drillbits.log:
> >
> > 2014-10-30 15:12:05,865 [3fb04943-c0e5-4801-a907-2bc3462a2997:frag:2:10] WARN o.a.d.e.w.fragment.FragmentExecutor - Failure while closing out resources.
> > java.lang.IllegalStateException: Failure while trying to close child allocator: Child level allocators not closed. Fragment 2:10. Stack trace:
> > org.apache.drill.exec.ops.OperatorContext.<init>(OperatorContext.java:41)
> > org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:90)
> > org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:148)
> > org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:54)
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan(AbstractPhysicalVisitor.java:113)
> > org.apache.drill.exec.store.parquet.ParquetRowGroupScan.accept(ParquetRowGroupScan.java:107)
> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
> > org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashAggregate(AbstractPhysicalVisitor.java:93)
> > org.apache.drill.exec.physical.config.HashAggregate.accept(HashAggregate.java:62)
> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
> > org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
> > org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:59)
> > org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSender(AbstractPhysicalVisitor.java:98)
> > org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashPartitionSender(AbstractPhysicalVisitor.java:141)
> > org.apache.drill.exec.physical.config.HashPartitionSender.accept(HashPartitionSender.java:66)
> > org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:87)
> > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:94)
> > org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > java.lang.Thread.run(Thread.java:745)
> > at org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:215) ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > at org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:271) ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > at org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:139) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:107) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_67]
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_67]
> >
> > Any help is greatly appreciated.
> >
> > Thanks & Regards
> > Tridib
> >
Re: Aggregation sum() fails on parquet file
Posted by Daniel Haviv <da...@gmail.com>.
Hi,
Any clue on that? I'm facing a similar problem
Thanks,
Daniel
> On 30 באוק׳ 2014, at 21:28, Tridib Samanta <tr...@live.com> wrote:
>
> Hello,
> I am running a query like: select sum(amount) from myhdfs.parquet.`claim.parquet`;
> This fails with the following error at sqlline:
>
> Query failed: Failure while running fragment. Schema is currently null. You must call buildSchema(SelectionVectorMode) before this container can return a schema. [00314ea1-342a-4322-ac00-f8307c554733]
> Error: exception while executing query: Failure while trying to get next result batch. (state=,code=0)
>
>
> Here is the stack trace from drillbits.log:
>
> 2014-10-30 15:12:05,865 [3fb04943-c0e5-4801-a907-2bc3462a2997:frag:2:10] WARN o.a.d.e.w.fragment.FragmentExecutor - Failure while closing out resources.
> java.lang.IllegalStateException: Failure while trying to close child allocator: Child level allocators not closed. Fragment 2:10. Stack trace:
> org.apache.drill.exec.ops.OperatorContext.<init>(OperatorContext.java:41)
> org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:90)
> org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:148)
> org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:54)
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan(AbstractPhysicalVisitor.java:113)
> org.apache.drill.exec.store.parquet.ParquetRowGroupScan.accept(ParquetRowGroupScan.java:107)
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashAggregate(AbstractPhysicalVisitor.java:93)
> org.apache.drill.exec.physical.config.HashAggregate.accept(HashAggregate.java:62)
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:196)
> org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:59)
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSender(AbstractPhysicalVisitor.java:98)
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitHashPartitionSender(AbstractPhysicalVisitor.java:141)
> org.apache.drill.exec.physical.config.HashPartitionSender.accept(HashPartitionSender.java:66)
> org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:87)
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:94)
> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:745)
> at org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:215) ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> at org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:271) ~[drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:139) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:107) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250) [drill-java-exec-0.5.0-incubating-rebuffed.jar:0.5.0-incubating]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_67]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_67]
>
> Any help is greatly appreciated.
>
> Thanks & Regards
> Tridib
>