You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by 何建军 <he...@163.com> on 2017/08/02 08:17:52 UTC
System Error PhysicalOperatorSetupException: Can not parallelize
fragment
Hi , nice to meet you .
thinks to read this email.
when used drill to Query, meets some error very confused me, and i don't know how to solve it ,hope you could check and help me thinks a lot .
to find any useful message ,so i modify some class to print error.
here is my modify :
drill-java-exec-1.9.0.jar
org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.java
line 80
from
checkOrThrow(endpointPool.size() <= width, logger,
"Number of mandatory endpoints ({}) that require an assignment is more than the allowed fragment max " +
"width ({}).", endpointPool.size(), pInfo.getMaxWidth());
to
checkOrThrow(endpointPool.size() <= width, logger,
"Number of mandatory endpoints ({}) that require an assignment is more than the allowed fragment max " +
"width ({}), fragment: {}, endpointPool: {}, pInfo.getEndpointAffinityMap(): {}.",
endpointPool.size(),
pInfo.getMaxWidth(),
fragmentWrapper.getNode(),
org.apache.commons.lang3.StringUtils.join( endpointPool.values(), ","),
org.apache.commons.lang3.StringUtils.join(pInfo.getEndpointAffinityMap().values(), ","));
here is the meesage of error log
[Error Id: 258f549e-d618-4dcb-994a-3e3908346033 on slave1:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: PhysicalOperatorSetupException: Can not parallelize fragment.
[Error Id: 258f549e-d618-4dcb-994a-3e3908346033 on slave1:31010]
at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543) ~[drill-common-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:825) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:935) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:281) [drill-java-exec-1.9.0.jar:1.9.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_121]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: Can not parallelize fragment.
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:282) [drill-java-exec-1.9.0.jar:1.9.0]
... 3 common frames omitted
Caused by: org.apache.drill.exec.physical.PhysicalOperatorSetupException: Can not parallelize fragment.
at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.checkOrThrow(HardAffinityFragmentParallelizer.java:162) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.parallelizeFragment(HardAffinityFragmentParallelizer.java:89) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.SimpleParallelizer.parallelizeFragment(SimpleParallelizer.java:253) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragmentsHelper(SimpleParallelizer.java:167) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragments(SimpleParallelizer.java:126) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.getQueryWorkUnit(Foreman.java:596) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.runPhysicalPlan(Foreman.java:426) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1010) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264) [drill-java-exec-1.9.0.jar:1.9.0]
... 3 common frames omitted
2017-07-27 12:01:15,912 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.drill.exec.work.foreman.Foreman - Query text for query id 26869a74-53da-2307-1e8d-9326be0d04eb: select sum(t.num) num from (select province,count(distinct phone) as num from indexr.count_phone where ctime <= 20170725 group by province) t
2017-07-27 12:01:16,100 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 6 classes for org.apache.drill.exec.store.dfs.FormatPlugin took 0ms
2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 7 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 7 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 7 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-07-27 12:01:16,117 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=: 1
2017-07-27 12:01:16,117 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=: 101
2017-07-27 12:01:16,139 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=: 1
2017-07-27 12:01:16,139 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=: 101
2017-07-27 12:01:16,142 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=: 1
2017-07-27 12:01:16,142 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=: 101
2017-07-27 12:01:16,153 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 33hjj=======================maxPw=: 1
2017-07-27 12:01:16,153 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] INFO o.a.d.e.s.indexr.ScanWrokProvider - 44hjj=======================maxPw=: 1001
2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] ERROR o.a.d.e.p.f.HardAffinityFragmentParallelizer - 000000============width=2
2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] ERROR o.a.d.e.p.f.HardAffinityFragmentParallelizer - 111111============width=1 pInfo.getMaxWidth()=1
2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] ERROR o.a.d.e.p.f.HardAffinityFragmentParallelizer - Number of mandatory endpoints (2) that require an assignment is more than the allowed fragment max width (1), fragment: FragmentNode [root=Screen [endpoint=address: "slave1"
user_port: 31010
control_port: 31011
data_port: 31012
, getChild()=org.apache.drill.exec.physical.config.Project@ae9bd17], sendingExchange=null, receivingExchangePairs=[]], endpointPool: EndpointAffinity [endpoint=address: "slave1" user_port: 31010 control_port: 31011 data_port: 31012, affinity=1.0, mandatory=true, maxWidth=1],EndpointAffinity [endpoint=address: "slave2" user_port: 31010 control_port: 31011 data_port: 31012, affinity=3616256.0, mandatory=true, maxWidth=2147483647], pInfo.getEndpointAffinityMap(): EndpointAffinity [endpoint=address: "slave1" user_port: 31010 control_port: 31011 data_port: 31012, affinity=1.0, mandatory=true, maxWidth=1],EndpointAffinity [endpoint=address: "slave2" user_port: 31010 control_port: 31011 data_port: 31012, affinity=3616256.0, mandatory=true, maxWidth=2147483647].
2017-07-27 12:01:16,160 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman] ERROR o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: PhysicalOperatorSetupException: Can not parallelize fragment.
[Error Id: 7d2c1de5-83c7-473e-aba4-8540c39e14d8 on slave1:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: PhysicalOperatorSetupException: Can not parallelize fragment.
[Error Id: 7d2c1de5-83c7-473e-aba4-8540c39e14d8 on slave1:31010]
at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543) ~[drill-common-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:825) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:935) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:281) [drill-java-exec-1.9.0.jar:1.9.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_121]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: Can not parallelize fragment.
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:282) [drill-java-exec-1.9.0.jar:1.9.0]
... 3 common frames omitted
Caused by: org.apache.drill.exec.physical.PhysicalOperatorSetupException: Can not parallelize fragment.
at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.checkOrThrow(HardAffinityFragmentParallelizer.java:162) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParallelizer.parallelizeFragment(HardAffinityFragmentParallelizer.java:89) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.SimpleParallelizer.parallelizeFragment(SimpleParallelizer.java:253) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragmentsHelper(SimpleParallelizer.java:167) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragments(SimpleParallelizer.java:126) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.getQueryWorkUnit(Foreman.java:596) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.runPhysicalPlan(Foreman.java:426) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1010) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264) [drill-java-exec-1.9.0.jar:1.9.0]
... 3 common frames omitted
Re: System Error PhysicalOperatorSetupException: Can not parallelize
fragment
Posted by Padma Penumarthy <pp...@mapr.com>.
Actually, the way parallelization works is documented in this JIRA,
https://issues.apache.org/jira/browse/DRILL-4706
Essentially, if we have screen and scan in the same fragment, affinity for the fragment will be hard. So, that is not the problem here. Problem seem to be in how the groupScan is implemented. Based on logs, both endpoints are marked as mandatory (is that needed ?) and at the same time, max parallelization seem to have been configured as 1 (getMaxParallelizationWidth() in groupScan).
Thanks,
Padma
________________________________
From: Jinfeng Ni <jn...@apache.org>
Sent: Wednesday, August 2, 2017 11:30 AM
To: user
Subject: Re: System Error PhysicalOperatorSetupException: Can not parallelize fragment
Are you querying a new storage plugin (I saw the stack trace shows
"indexR"), and that storage plugin is using
HardAffinityFragmentParallelizer? As far as I know, the hard assignment is
only used for distributed system tables, or some special operator (Screen)
in Drill codebase. The majority cases use soft one. Hard assignment means
to assign at least one minor fragment to each available drill endpoint.
For instance, screen has to assign to the foreman node, while distributed
system tables requires at least one minor fragment for each drillbit.
In your case, seems you have two drillbits, while the fragment # is only 1.
That fails the requirement for hard assignment. Either you choose to use
soft assignment, or make # of drillbits <= # of fragments.
On Wed, Aug 2, 2017 at 1:17 AM, 何建军 <he...@163.com> wrote:
> Hi , nice to meet you .
> thinks to read this email.
> when used drill to Query, meets some error very confused me, and i don't
> know how to solve it ,hope you could check and help me thinks a lot .
>
>
> to find any useful message ,so i modify some class to print error.
> here is my modify :
> drill-java-exec-1.9.0.jar
> org.apache.drill.exec.planner.fragment.HardAffinityFragmentParalleliz
> er.java
> line 80
> from
> checkOrThrow(endpointPool.size() <= width, logger,
> "Number of mandatory endpoints ({}) that require an assignment is
> more than the allowed fragment max " +
> "width ({}).", endpointPool.size(), pInfo.getMaxWidth());
> to
> checkOrThrow(endpointPool.size() <= width, logger,
> "Number of mandatory endpoints ({}) that require an assignment
> is more than the allowed fragment max " +
> "width ({}), fragment: {}, endpointPool: {},
> pInfo.getEndpointAffinityMap(): {}.",
> endpointPool.size(),
> pInfo.getMaxWidth(),
> fragmentWrapper.getNode(),
> org.apache.commons.lang3.StringUtils.join(
> endpointPool.values(), ","),
> org.apache.commons.lang3.StringUtils.join(pInfo.
> getEndpointAffinityMap().values(), ","));
>
>
> here is the meesage of error log
>
>
>
>
> [Error Id: 258f549e-d618-4dcb-994a-3e3908346033 on slave1:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> PhysicalOperatorSetupException: Can not parallelize fragment.
>
>
>
>
> [Error Id: 258f549e-d618-4dcb-994a-3e3908346033 on slave1:31010]
> at org.apache.drill.common.exceptions.UserException$
> Builder.build(UserException.java:543) ~[drill-common-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:825)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:935)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:281)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> Caused by: org.apache.drill.exec.work.foreman.ForemanException:
> Unexpected exception during fragment initialization: Can not parallelize
> fragment.
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:282)
> [drill-java-exec-1.9.0.jar:1.9.0]
> ... 3 common frames omitted
> Caused by: org.apache.drill.exec.physical.PhysicalOperatorSetupException:
> Can not parallelize fragment.
> at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParalleliz
> er.checkOrThrow(HardAffinityFragmentParallelizer.java:162)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParalleliz
> er.parallelizeFragment(HardAffinityFragmentParallelizer.java:89)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.SimpleParallelizer.
> parallelizeFragment(SimpleParallelizer.java:253)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.SimpleParallelizer.
> getFragmentsHelper(SimpleParallelizer.java:167)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragments(SimpleParallelizer.java:126)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.
> getQueryWorkUnit(Foreman.java:596) [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.
> runPhysicalPlan(Foreman.java:426) [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1010)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264)
> [drill-java-exec-1.9.0.jar:1.9.0]
> ... 3 common frames omitted
> 2017-07-27 12:01:15,912 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.drill.exec.work.foreman.Foreman - Query text for query id
> 26869a74-53da-2307-1e8d-9326be0d04eb: select sum(t.num) num from (select
> province,count(distinct phone) as num from indexr.count_phone where ctime
> <= 20170725 group by province) t
> 2017-07-27 12:01:16,100 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.c.s.persistence.ScanResult - loading 6 classes for
> org.apache.drill.exec.store.dfs.FormatPlugin took 0ms
> 2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.c.s.persistence.ScanResult - loading 7 classes for
> org.apache.drill.common.logical.FormatPluginConfig took 0ms
> 2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.c.s.persistence.ScanResult - loading 7 classes for
> org.apache.drill.common.logical.FormatPluginConfig took 0ms
> 2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.c.s.persistence.ScanResult - loading 7 classes for
> org.apache.drill.common.logical.FormatPluginConfig took 0ms
> 2017-07-27 12:01:16,117 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=:
> 1
> 2017-07-27 12:01:16,117 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=:
> 101
> 2017-07-27 12:01:16,139 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=:
> 1
> 2017-07-27 12:01:16,139 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=:
> 101
> 2017-07-27 12:01:16,142 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=:
> 1
> 2017-07-27 12:01:16,142 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=:
> 101
> 2017-07-27 12:01:16,153 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 33hjj=======================maxPw=:
> 1
> 2017-07-27 12:01:16,153 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 44hjj=======================maxPw=:
> 1001
> 2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> ERROR o.a.d.e.p.f.HardAffinityFragmentParallelizer -
> 000000============width=2
> 2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> ERROR o.a.d.e.p.f.HardAffinityFragmentParallelizer -
> 111111============width=1 pInfo.getMaxWidth()=1
> 2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> ERROR o.a.d.e.p.f.HardAffinityFragmentParallelizer - Number of mandatory
> endpoints (2) that require an assignment is more than the allowed fragment
> max width (1), fragment: FragmentNode [root=Screen [endpoint=address:
> "slave1"
> user_port: 31010
> control_port: 31011
> data_port: 31012
> , getChild()=org.apache.drill.exec.physical.config.Project@ae9bd17],
> sendingExchange=null, receivingExchangePairs=[]], endpointPool:
> EndpointAffinity [endpoint=address: "slave1" user_port: 31010 control_port:
> 31011 data_port: 31012, affinity=1.0, mandatory=true,
> maxWidth=1],EndpointAffinity [endpoint=address: "slave2" user_port: 31010
> control_port: 31011 data_port: 31012, affinity=3616256.0, mandatory=true,
> maxWidth=2147483647], pInfo.getEndpointAffinityMap(): EndpointAffinity
> [endpoint=address: "slave1" user_port: 31010 control_port: 31011 data_port:
> 31012, affinity=1.0, mandatory=true, maxWidth=1],EndpointAffinity
> [endpoint=address: "slave2" user_port: 31010 control_port: 31011 data_port:
> 31012, affinity=3616256.0, mandatory=true, maxWidth=2147483647].
> 2017-07-27 12:01:16,160 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> ERROR o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR:
> PhysicalOperatorSetupException: Can not parallelize fragment.
>
>
>
>
> [Error Id: 7d2c1de5-83c7-473e-aba4-8540c39e14d8 on slave1:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> PhysicalOperatorSetupException: Can not parallelize fragment.
>
>
>
>
> [Error Id: 7d2c1de5-83c7-473e-aba4-8540c39e14d8 on slave1:31010]
> at org.apache.drill.common.exceptions.UserException$
> Builder.build(UserException.java:543) ~[drill-common-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:825)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:935)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:281)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> Caused by: org.apache.drill.exec.work.foreman.ForemanException:
> Unexpected exception during fragment initialization: Can not parallelize
> fragment.
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:282)
> [drill-java-exec-1.9.0.jar:1.9.0]
> ... 3 common frames omitted
> Caused by: org.apache.drill.exec.physical.PhysicalOperatorSetupException:
> Can not parallelize fragment.
> at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParalleliz
> er.checkOrThrow(HardAffinityFragmentParallelizer.java:162)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParalleliz
> er.parallelizeFragment(HardAffinityFragmentParallelizer.java:89)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.SimpleParallelizer.
> parallelizeFragment(SimpleParallelizer.java:253)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.SimpleParallelizer.
> getFragmentsHelper(SimpleParallelizer.java:167)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragments(SimpleParallelizer.java:126)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.
> getQueryWorkUnit(Foreman.java:596) [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.
> runPhysicalPlan(Foreman.java:426) [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1010)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264)
> [drill-java-exec-1.9.0.jar:1.9.0]
> ... 3 common frames omitted
>
>
>
>
Re: System Error PhysicalOperatorSetupException: Can not parallelize fragment
Posted by Jinfeng Ni <jn...@apache.org>.
Are you querying a new storage plugin (I saw the stack trace shows
"indexR"), and that storage plugin is using
HardAffinityFragmentParallelizer? As far as I know, the hard assignment is
only used for distributed system tables, or some special operator (Screen)
in Drill codebase. The majority cases use soft one. Hard assignment means
to assign at least one minor fragment to each available drill endpoint.
For instance, screen has to assign to the foreman node, while distributed
system tables requires at least one minor fragment for each drillbit.
In your case, seems you have two drillbits, while the fragment # is only 1.
That fails the requirement for hard assignment. Either you choose to use
soft assignment, or make # of drillbits <= # of fragments.
On Wed, Aug 2, 2017 at 1:17 AM, 何建军 <he...@163.com> wrote:
> Hi , nice to meet you .
> thinks to read this email.
> when used drill to Query, meets some error very confused me, and i don't
> know how to solve it ,hope you could check and help me thinks a lot .
>
>
> to find any useful message ,so i modify some class to print error.
> here is my modify :
> drill-java-exec-1.9.0.jar
> org.apache.drill.exec.planner.fragment.HardAffinityFragmentParalleliz
> er.java
> line 80
> from
> checkOrThrow(endpointPool.size() <= width, logger,
> "Number of mandatory endpoints ({}) that require an assignment is
> more than the allowed fragment max " +
> "width ({}).", endpointPool.size(), pInfo.getMaxWidth());
> to
> checkOrThrow(endpointPool.size() <= width, logger,
> "Number of mandatory endpoints ({}) that require an assignment
> is more than the allowed fragment max " +
> "width ({}), fragment: {}, endpointPool: {},
> pInfo.getEndpointAffinityMap(): {}.",
> endpointPool.size(),
> pInfo.getMaxWidth(),
> fragmentWrapper.getNode(),
> org.apache.commons.lang3.StringUtils.join(
> endpointPool.values(), ","),
> org.apache.commons.lang3.StringUtils.join(pInfo.
> getEndpointAffinityMap().values(), ","));
>
>
> here is the meesage of error log
>
>
>
>
> [Error Id: 258f549e-d618-4dcb-994a-3e3908346033 on slave1:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> PhysicalOperatorSetupException: Can not parallelize fragment.
>
>
>
>
> [Error Id: 258f549e-d618-4dcb-994a-3e3908346033 on slave1:31010]
> at org.apache.drill.common.exceptions.UserException$
> Builder.build(UserException.java:543) ~[drill-common-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:825)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:935)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:281)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> Caused by: org.apache.drill.exec.work.foreman.ForemanException:
> Unexpected exception during fragment initialization: Can not parallelize
> fragment.
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:282)
> [drill-java-exec-1.9.0.jar:1.9.0]
> ... 3 common frames omitted
> Caused by: org.apache.drill.exec.physical.PhysicalOperatorSetupException:
> Can not parallelize fragment.
> at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParalleliz
> er.checkOrThrow(HardAffinityFragmentParallelizer.java:162)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParalleliz
> er.parallelizeFragment(HardAffinityFragmentParallelizer.java:89)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.SimpleParallelizer.
> parallelizeFragment(SimpleParallelizer.java:253)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.SimpleParallelizer.
> getFragmentsHelper(SimpleParallelizer.java:167)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragments(SimpleParallelizer.java:126)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.
> getQueryWorkUnit(Foreman.java:596) [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.
> runPhysicalPlan(Foreman.java:426) [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1010)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264)
> [drill-java-exec-1.9.0.jar:1.9.0]
> ... 3 common frames omitted
> 2017-07-27 12:01:15,912 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.drill.exec.work.foreman.Foreman - Query text for query id
> 26869a74-53da-2307-1e8d-9326be0d04eb: select sum(t.num) num from (select
> province,count(distinct phone) as num from indexr.count_phone where ctime
> <= 20170725 group by province) t
> 2017-07-27 12:01:16,100 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.c.s.persistence.ScanResult - loading 6 classes for
> org.apache.drill.exec.store.dfs.FormatPlugin took 0ms
> 2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.c.s.persistence.ScanResult - loading 7 classes for
> org.apache.drill.common.logical.FormatPluginConfig took 0ms
> 2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.c.s.persistence.ScanResult - loading 7 classes for
> org.apache.drill.common.logical.FormatPluginConfig took 0ms
> 2017-07-27 12:01:16,101 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.c.s.persistence.ScanResult - loading 7 classes for
> org.apache.drill.common.logical.FormatPluginConfig took 0ms
> 2017-07-27 12:01:16,117 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=:
> 1
> 2017-07-27 12:01:16,117 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=:
> 101
> 2017-07-27 12:01:16,139 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=:
> 1
> 2017-07-27 12:01:16,139 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=:
> 101
> 2017-07-27 12:01:16,142 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 0hjj=======================maxPw=:
> 1
> 2017-07-27 12:01:16,142 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 1hjj=======================maxPw=:
> 101
> 2017-07-27 12:01:16,153 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 33hjj=======================maxPw=:
> 1
> 2017-07-27 12:01:16,153 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> INFO o.a.d.e.s.indexr.ScanWrokProvider - 44hjj=======================maxPw=:
> 1001
> 2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> ERROR o.a.d.e.p.f.HardAffinityFragmentParallelizer -
> 000000============width=2
> 2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> ERROR o.a.d.e.p.f.HardAffinityFragmentParallelizer -
> 111111============width=1 pInfo.getMaxWidth()=1
> 2017-07-27 12:01:16,154 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> ERROR o.a.d.e.p.f.HardAffinityFragmentParallelizer - Number of mandatory
> endpoints (2) that require an assignment is more than the allowed fragment
> max width (1), fragment: FragmentNode [root=Screen [endpoint=address:
> "slave1"
> user_port: 31010
> control_port: 31011
> data_port: 31012
> , getChild()=org.apache.drill.exec.physical.config.Project@ae9bd17],
> sendingExchange=null, receivingExchangePairs=[]], endpointPool:
> EndpointAffinity [endpoint=address: "slave1" user_port: 31010 control_port:
> 31011 data_port: 31012, affinity=1.0, mandatory=true,
> maxWidth=1],EndpointAffinity [endpoint=address: "slave2" user_port: 31010
> control_port: 31011 data_port: 31012, affinity=3616256.0, mandatory=true,
> maxWidth=2147483647], pInfo.getEndpointAffinityMap(): EndpointAffinity
> [endpoint=address: "slave1" user_port: 31010 control_port: 31011 data_port:
> 31012, affinity=1.0, mandatory=true, maxWidth=1],EndpointAffinity
> [endpoint=address: "slave2" user_port: 31010 control_port: 31011 data_port:
> 31012, affinity=3616256.0, mandatory=true, maxWidth=2147483647].
> 2017-07-27 12:01:16,160 [26869a74-53da-2307-1e8d-9326be0d04eb:foreman]
> ERROR o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR:
> PhysicalOperatorSetupException: Can not parallelize fragment.
>
>
>
>
> [Error Id: 7d2c1de5-83c7-473e-aba4-8540c39e14d8 on slave1:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> PhysicalOperatorSetupException: Can not parallelize fragment.
>
>
>
>
> [Error Id: 7d2c1de5-83c7-473e-aba4-8540c39e14d8 on slave1:31010]
> at org.apache.drill.common.exceptions.UserException$
> Builder.build(UserException.java:543) ~[drill-common-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:825)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:935)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:281)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_121]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> Caused by: org.apache.drill.exec.work.foreman.ForemanException:
> Unexpected exception during fragment initialization: Can not parallelize
> fragment.
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:282)
> [drill-java-exec-1.9.0.jar:1.9.0]
> ... 3 common frames omitted
> Caused by: org.apache.drill.exec.physical.PhysicalOperatorSetupException:
> Can not parallelize fragment.
> at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParalleliz
> er.checkOrThrow(HardAffinityFragmentParallelizer.java:162)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.HardAffinityFragmentParalleliz
> er.parallelizeFragment(HardAffinityFragmentParallelizer.java:89)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.SimpleParallelizer.
> parallelizeFragment(SimpleParallelizer.java:253)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.SimpleParallelizer.
> getFragmentsHelper(SimpleParallelizer.java:167)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragments(SimpleParallelizer.java:126)
> ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.
> getQueryWorkUnit(Foreman.java:596) [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.
> runPhysicalPlan(Foreman.java:426) [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1010)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264)
> [drill-java-exec-1.9.0.jar:1.9.0]
> ... 3 common frames omitted
>
>
>
>