You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@iotdb.apache.org by "刘珍 (Jira)" <ji...@apache.org> on 2023/03/23 02:42:00 UTC
[jira] [Commented] (IOTDB-5716) Wrong dependency when pipeline consumeOneByOneOperator
[ https://issues.apache.org/jira/browse/IOTDB-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703836#comment-17703836 ]
刘珍 commented on IOTDB-5716:
---------------------------
测试流程
1. 启动1C1D
测试版本：master_0321_8441842
192.168.10.76 /data/mpp_test/m_0321_8441842
需要显式设置的参数
CN:
MAX_HEAP_SIZE="8G"
DN：
MAX_HEAP_SIZE="256G"
MAX_DIRECT_MEMORY_SIZE="32G"
2.bm_dir下启动脚本run.sh
3. iotdb_dir下启动select_into.sh
4.查询数据
select count(s_0) from root.** align by device
报错Msg: 301: The system can't allow more query tasks.
日志：
2023-03-21 17:30:47,267 [pool-25-IoTDB-ClientRPC-Processor-1$20230321_093021_21808_1.2.0] WARN o.a.i.d.m.e.f.FragmentInstanceManager:154 - error when create FragmentInstanceExecution.
java.lang.IllegalStateException: The system can't allow more query tasks.
at com.google.common.base.Preconditions.checkState(Preconditions.java:502)
at org.apache.iotdb.db.mpp.execution.schedule.queue.IndexedBlockingReserveQueue.push(IndexedBlockingReserveQueue.java:57)
at org.apache.iotdb.db.mpp.execution.schedule.DriverScheduler.submitTaskToReadyQueue(DriverScheduler.java:246)
at org.apache.iotdb.db.mpp.execution.schedule.DriverScheduler.submitDrivers(DriverScheduler.java:220)
at org.apache.iotdb.db.mpp.execution.fragment.FragmentInstanceExecution.createFragmentInstanceExecution(FragmentInstanceExecution.java:65)
at org.apache.iotdb.db.mpp.execution.fragment.FragmentInstanceManager.lambda$execDataQueryFragmentInstance$2(FragmentInstanceManager.java:144)
at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705)
at org.apache.iotdb.db.mpp.execution.fragment.FragmentInstanceManager.execDataQueryFragmentInstance(FragmentInstanceManager.java:115)
at org.apache.iotdb.db.consensus.statemachine.DataRegionStateMachine.read(DataRegionStateMachine.java:265)
at org.apache.iotdb.consensus.iot.IoTConsensusServerImpl.read(IoTConsensusServerImpl.java:298)
at org.apache.iotdb.consensus.iot.IoTConsensus.read(IoTConsensus.java:184)
at org.apache.iotdb.db.mpp.execution.executor.RegionReadExecutor.execute(RegionReadExecutor.java:46)
at org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchLocally(FragmentInstanceDispatcherImpl.java:335)
at org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchOneInstance(FragmentInstanceDispatcherImpl.java:235)
at org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchRead(FragmentInstanceDispatcherImpl.java:113)
at org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatch(FragmentInstanceDispatcherImpl.java:100)
at org.apache.iotdb.db.mpp.plan.scheduler.ClusterScheduler.start(ClusterScheduler.java:116)
at org.apache.iotdb.db.mpp.plan.execution.QueryExecution.schedule(QueryExecution.java:311)
at org.apache.iotdb.db.mpp.plan.execution.QueryExecution.start(QueryExecution.java:215)
at org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:161)
at org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.executeStatementInternal(ClientRPCServiceImpl.java:218)
at org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.executeStatementV2(ClientRPCServiceImpl.java:480)
at org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$executeStatementV2.getResult(IClientRPCService.java:3629)
at org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$executeStatementV2.getResult(IClientRPCService.java:3609)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at org.apache.iotdb.db.service.thrift.ProcessorWithMetrics.process(ProcessorWithMetrics.java:64)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
2023-03-21 17:30:47,269 [pool-25-IoTDB-ClientRPC-Processor-1$20230321_093021_21808_1.2.0] WARN o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:337 - The system can't allow more query tasks.
2023-03-21 17:30:47,287 [pool-25-IoTDB-ClientRPC-Processor-1$20230321_093021_21808_1] INFO o.a.i.d.m.p.Coordinator:221 - Cost: 25877 ms, sql is select count(s_0) from root.** align by device
> Wrong dependency when pipeline consumeOneByOneOperator
> ------------------------------------------------------
>
> Key: IOTDB-5716
> URL: https://issues.apache.org/jira/browse/IOTDB-5716
> Project: Apache IoTDB
> Issue Type: Bug
> Reporter: Alima777
> Assignee: Alima777
> Priority: Major
> Labels: pull-request-available
>
> For example,
> DeviceViewOperator
> * AggregationOperator1
> ** SeriesAggregationScanOperator1
> ** ExchangeOperator1
> * AggregationOperator2
> ** SeriesAggregationScanOperator2
> ** ExchangeOperator2
> If dop = 3, AggregationOperator1 and SeriesAggregationScanOperator1
> will both be a pipeline. Howerver, due to the traverseorder, the first pipeline of this group will be the SeriesAggregationScanOperator1 rather than AggregationOperator1.
> In before logic, I took the first pipeline as the dependencyPipeline.
> But actually AggregationOperator2 and SeriesAggregationScanOperator2 must have dependency on the parent pipeline, i.e. pipeline of AggregationOperator1.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)