You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Kasia Kucharczyk (JIRA)" <ji...@apache.org> on 2019/06/06 11:46:00 UTC
[jira] [Created] (BEAM-7497) SyntheticSource option
forceNumInitialBundles is not optional in all cases
Kasia Kucharczyk created BEAM-7497:
--------------------------------------
Summary: SyntheticSource option forceNumInitialBundles is not optional in all cases
Key: BEAM-7497
URL: https://issues.apache.org/jira/browse/BEAM-7497
Project: Beam
Issue Type: Bug
Components: testing
Reporter: Kasia Kucharczyk
It fails when numRecords is between 1 and 3.
I noticed it by running SyntheticDataPublisher:
{code:java}
./gradlew :beam-sdks-java-load-tests:run -PloadTest.args='--pubSubTopic=projects/apache-beam-io-testing/topics/loadtests --sourceOptions={"numRecords":2}' -PloadTest.mainClass="org.apache.beam.sdk.loadtests.SyntheticDataPublisher"{code}
The exception:
{code:java}
Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at org.apache.beam.sdk.io.synthetic.BundleSplitter.getRelativeBundleSizes(BundleSplitter.java:60)
at org.apache.beam.sdk.io.synthetic.BundleSplitter.getBundleSizes(BundleSplitter.java:39)
at org.apache.beam.sdk.io.synthetic.SyntheticBoundedSource.split(SyntheticBoundedSource.java:146)
at org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$InputProvider.getInitialInputs(BoundedReadEvaluatorFactory.java:214)
at org.apache.beam.runners.direct.ReadEvaluatorFactory$InputProvider.getInitialInputs(ReadEvaluatorFactory.java:89)
at org.apache.beam.runners.direct.RootProviderRegistry.getInitialInputs(RootProviderRegistry.java:76)
at org.apache.beam.runners.direct.ExecutorServiceParallelExecutor.start(ExecutorServiceParallelExecutor.java:155)
at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:208)
at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:67)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
at org.apache.beam.sdk.loadtests.SyntheticDataPublisher.main(SyntheticDataPublisher.java:150)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)