You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Sorabh Hamirwasia (JIRA)" <ji...@apache.org> on 2019/04/11 19:05:00 UTC
[jira] [Created] (DRILL-7170) IllegalStateException: Record count
not set for this vector container
Sorabh Hamirwasia created DRILL-7170:
----------------------------------------
Summary: IllegalStateException: Record count not set for this vector container
Key: DRILL-7170
URL: https://issues.apache.org/jira/browse/DRILL-7170
Project: Apache Drill
Issue Type: Bug
Components: Execution - Relational Operators
Reporter: Sorabh Hamirwasia
Fix For: 1.17.0
{code:java}
Query: /root/drillAutomation/master/framework/resources/Advanced/tpcds/tpcds_sf1/original/maprdb/json/query95.sql
WITH ws_wh AS
(
SELECT ws1.ws_order_number,
ws1.ws_warehouse_sk wh1,
ws2.ws_warehouse_sk wh2
FROM web_sales ws1,
web_sales ws2
WHERE ws1.ws_order_number = ws2.ws_order_number
AND ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk)
SELECT
Count(DISTINCT ws_order_number) AS `order count` ,
Sum(ws_ext_ship_cost) AS `total shipping cost` ,
Sum(ws_net_profit) AS `total net profit`
FROM web_sales ws1 ,
date_dim ,
customer_address ,
web_site
WHERE d_date BETWEEN '2000-04-01' AND (
Cast('2000-04-01' AS DATE) + INTERVAL '60' day)
AND ws1.ws_ship_date_sk = d_date_sk
AND ws1.ws_ship_addr_sk = ca_address_sk
AND ca_state = 'IN'
AND ws1.ws_web_site_sk = web_site_sk
AND web_company_name = 'pri'
AND ws1.ws_order_number IN
(
SELECT ws_order_number
FROM ws_wh)
AND ws1.ws_order_number IN
(
SELECT wr_order_number
FROM web_returns,
ws_wh
WHERE wr_order_number = ws_wh.ws_order_number)
ORDER BY count(DISTINCT ws_order_number)
LIMIT 100
Exception:
java.sql.SQLException: SYSTEM ERROR: IllegalStateException: Record count not set for this vector container
Fragment 2:3
Please, refer to logs for more information.
[Error Id: 4ed92fce-505b-40ba-ac0e-4a302c28df47 on drill87:31010]
(java.lang.IllegalStateException) Record count not set for this vector container
org.apache.drill.shaded.guava.com.google.common.base.Preconditions.checkState():459
org.apache.drill.exec.record.VectorContainer.getRecordCount():394
org.apache.drill.exec.record.RecordBatchSizer.<init>():720
org.apache.drill.exec.record.RecordBatchSizer.<init>():704
org.apache.drill.exec.physical.impl.common.HashTableTemplate$BatchHolder.getActualSize():462
org.apache.drill.exec.physical.impl.common.HashTableTemplate.getActualSize():964
org.apache.drill.exec.physical.impl.common.HashTableTemplate.makeDebugString():973
org.apache.drill.exec.physical.impl.common.HashPartition.makeDebugString():601
org.apache.drill.exec.physical.impl.join.HashJoinBatch.makeDebugString():1313
org.apache.drill.exec.physical.impl.join.HashJoinBatch.executeBuildPhase():1105
org.apache.drill.exec.physical.impl.join.HashJoinBatch.innerNext():525
org.apache.drill.exec.record.AbstractRecordBatch.next():186
org.apache.drill.exec.record.AbstractRecordBatch.next():126
org.apache.drill.exec.record.AbstractRecordBatch.next():116
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():141
org.apache.drill.exec.record.AbstractRecordBatch.next():186
org.apache.drill.exec.record.AbstractRecordBatch.next():126
org.apache.drill.exec.test.generated.HashAggregatorGen1068899.doWork():642
org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.innerNext():296
org.apache.drill.exec.record.AbstractRecordBatch.next():186
org.apache.drill.exec.record.AbstractRecordBatch.next():126
org.apache.drill.exec.record.AbstractRecordBatch.next():116
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():141
org.apache.drill.exec.record.AbstractRecordBatch.next():186
org.apache.drill.exec.physical.impl.BaseRootExec.next():104
org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext():93
org.apache.drill.exec.physical.impl.BaseRootExec.next():94
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():296
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():283
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():422
org.apache.hadoop.security.UserGroupInformation.doAs():1669
org.apache.drill.exec.work.fragment.FragmentExecutor.run():283
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1149
java.util.concurrent.ThreadPoolExecutor$Worker.run():624
java.lang.Thread.run():748
at org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:538)
at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:642)
at oadd.org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:217)
at org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:148)
at org.apache.drill.test.framework.DrillTestJdbc.executeQuery(DrillTestJdbc.java:254)
at org.apache.drill.test.framework.DrillTestJdbc.run(DrillTestJdbc.java:115)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: oadd.org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: IllegalStateException: Record count not set for this vector container
Fragment 2:3
Please, refer to logs for more information.
[Error Id: 4ed92fce-505b-40ba-ac0e-4a302c28df47 on drill87:31010]
(java.lang.IllegalStateException) Record count not set for this vector container
org.apache.drill.shaded.guava.com.google.common.base.Preconditions.checkState():459
org.apache.drill.exec.record.VectorContainer.getRecordCount():394
org.apache.drill.exec.record.RecordBatchSizer.<init>():720
org.apache.drill.exec.record.RecordBatchSizer.<init>():704
org.apache.drill.exec.physical.impl.common.HashTableTemplate$BatchHolder.getActualSize():462
org.apache.drill.exec.physical.impl.common.HashTableTemplate.getActualSize():964
org.apache.drill.exec.physical.impl.common.HashTableTemplate.makeDebugString():973
org.apache.drill.exec.physical.impl.common.HashPartition.makeDebugString():601
org.apache.drill.exec.physical.impl.join.HashJoinBatch.makeDebugString():1313
org.apache.drill.exec.physical.impl.join.HashJoinBatch.executeBuildPhase():1105
org.apache.drill.exec.physical.impl.join.HashJoinBatch.innerNext():525
org.apache.drill.exec.record.AbstractRecordBatch.next():186
org.apache.drill.exec.record.AbstractRecordBatch.next():126
org.apache.drill.exec.record.AbstractRecordBatch.next():116
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():141
org.apache.drill.exec.record.AbstractRecordBatch.next():186
org.apache.drill.exec.record.AbstractRecordBatch.next():126
org.apache.drill.exec.test.generated.HashAggregatorGen1068899.doWork():642
org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.innerNext():296
org.apache.drill.exec.record.AbstractRecordBatch.next():186
org.apache.drill.exec.record.AbstractRecordBatch.next():126
org.apache.drill.exec.record.AbstractRecordBatch.next():116
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():141
org.apache.drill.exec.record.AbstractRecordBatch.next():186
org.apache.drill.exec.physical.impl.BaseRootExec.next():104
org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext():93
org.apache.drill.exec.physical.impl.BaseRootExec.next():94
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():296
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():283
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():422
org.apache.hadoop.security.UserGroupInformation.doAs():1669
org.apache.drill.exec.work.fragment.FragmentExecutor.run():283
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1149
java.util.concurrent.ThreadPoolExecutor$Worker.run():624
java.lang.Thread.run():748
at oadd.org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123)
at oadd.org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:422)
at oadd.org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:96)
at oadd.org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:273)
at oadd.org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:243)
at oadd.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at oadd.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at oadd.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at oadd.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312)
at oadd.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286)
at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at oadd.io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at oadd.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294)
at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at oadd.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911)
at oadd.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at oadd.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
at oadd.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
at oadd.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
at oadd.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
at oadd.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
... 1 more
Caused by: java.lang.IllegalStateException: Record count not set for this vector container
at org.apache.drill.shaded.guava.com.google.common.base.Preconditions.checkState(Preconditions.java:459)
at org.apache.drill.exec.record.VectorContainer.getRecordCount(VectorContainer.java:394)
at org.apache.drill.exec.record.RecordBatchSizer.<init>(RecordBatchSizer.java:720)
at org.apache.drill.exec.record.RecordBatchSizer.<init>(RecordBatchSizer.java:704)
at org.apache.drill.exec.physical.impl.common.HashTableTemplate$BatchHolder.getActualSize(HashTableTemplate.java:462)
at org.apache.drill.exec.physical.impl.common.HashTableTemplate.getActualSize(HashTableTemplate.java:964)
at org.apache.drill.exec.physical.impl.common.HashTableTemplate.makeDebugString(HashTableTemplate.java:973)
at org.apache.drill.exec.physical.impl.common.HashPartition.makeDebugString(HashPartition.java:601)
at org.apache.drill.exec.physical.impl.join.HashJoinBatch.makeDebugString(HashJoinBatch.java:1313)
at org.apache.drill.exec.physical.impl.join.HashJoinBatch.executeBuildPhase(HashJoinBatch.java:1105)
at org.apache.drill.exec.physical.impl.join.HashJoinBatch.innerNext(HashJoinBatch.java:525)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:126)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:116)
at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63)
at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:141)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:126)
at org.apache.drill.exec.test.generated.HashAggregatorGen1068899.doWork(HashAggTemplate.java:642)
at org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.innerNext(HashAggBatch.java:296)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:126)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:116)
at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63)
at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:141)
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186)
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
at org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93)
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296)
at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283)
at .......(:0)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:283)
at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
at .......(:0)
{code}
When HashJoinBatch hits OOM condition while executing build phase then it catches OOMException and try to generate debugString which internally causes this IllegalStateException.
https://github.com/apache/drill/blob/1.16.0/exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java#L1105
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)