You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@storm.apache.org by "Ethan Li (Jira)" <ji...@apache.org> on 2020/07/06 22:10:00 UTC
[jira] [Updated] (STORM-3667) Exception message on submitting
topology due to incorrect worker max heap size should be more
straightforward
[ https://issues.apache.org/jira/browse/STORM-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ethan Li updated STORM-3667:
----------------------------
Summary: Exception message on submitting topology due to incorrect worker max heap size should be more straightforward (was: Exception on submitting topology due to incorrect worker max heap size should be more straightforward )
> Exception message on submitting topology due to incorrect worker max heap size should be more straightforward
> --------------------------------------------------------------------------------------------------------------
>
> Key: STORM-3667
> URL: https://issues.apache.org/jira/browse/STORM-3667
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Ethan Li
> Assignee: Ethan Li
> Priority: Minor
>
> Submitting a topology can fail when the largest component heap size is larger than
> {code:java}
> topology.worker.max.heap.size.mb
> {code}
> For example, when you submit a topology with the following command
> {code:java}
> bin/storm jar /home/y/lib64/jars/storm-starter.jar org.apache.storm.starter.WordCountTopology -c topology.worker.max.heap.size.mb=100 -c topology.component.resources.onheap.memory.mb=120 wc
> {code}
> While the nimbus log shows
> {code:java}
> 2020-07-06 21:51:12.026 o.a.s.d.n.Nimbus pool-29-thread-45 [WARN] Topology submission exception. (topology name='wc')
> java.lang.IllegalArgumentException: Topology will not be able to be successfully scheduled: Config TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB=100.0 < 120.0 (Largest memory requirement of a component in the topology). Perhaps set TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB to a larger amount
> at org.apache.storm.daemon.nimbus.Nimbus.validateTopologyWorkerMaxHeapSizeConfigs(Nimbus.java:1650) ~[storm-server-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT]
> at org.apache.storm.daemon.nimbus.Nimbus.submitTopologyWithOpts(Nimbus.java:3145) [storm-server-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT]
> at org.apache.storm.daemon.nimbus.Nimbus.submitTopology(Nimbus.java:3107) [storm-server-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT]
> {code}
> the exception returned to users is
> {code:java}
> 21:51:12.012 [main] INFO o.a.s.StormSubmitter - Submitting topology wc in distributed mode with conf {"storm.zookeeper.topology.auth.scheme":"digest","storm.zookeeper.topology.auth.payload":"-8967725949793348049:-5020411463316947973","topology.workers":3,"topology.component.resources.onheap.memory.mb":120,"topology.worker.max.heap.size.mb":100,"topology.debug":true}
> java.lang.RuntimeException: org.apache.storm.thrift.TApplicationException: Internal error processing submitTopology
> at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:291)
> at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:210)
> at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:173)
> at org.apache.storm.topology.ConfigurableTopology.submit(ConfigurableTopology.java:119)
> at org.apache.storm.starter.WordCountTopology.run(WordCountTopology.java:58)
> at org.apache.storm.topology.ConfigurableTopology.start(ConfigurableTopology.java:68)
> at org.apache.storm.starter.WordCountTopology.main(WordCountTopology.java:36)
> Caused by: org.apache.storm.thrift.TApplicationException: Internal error processing submitTopology
> at org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> at org.apache.storm.generated.Nimbus$Client.recv_submitTopology(Nimbus.java:319)
> at org.apache.storm.generated.Nimbus$Client.submitTopology(Nimbus.java:303)
> at org.apache.storm.StormSubmitter.submitTopologyInDistributeMode(StormSubmitter.java:344)
> at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:279)
> ... 6 more
> {code}
> which is not very helpful.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)