You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Michele Bertoni <mi...@mail.polimi.it> on 2015/05/10 19:18:44 UTC

Memory exception

Hi everybody,
I am running a flink instance on my IDE
sometimes (totally random) I start it i getting this error

Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
	at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$receiveWithLogMessages$1.applyOrElse(JobManager.scala:306)
	at scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
	at scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
	at scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
	at org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:37)
	at org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:30)
	at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118)
	at org.apache.flink.runtime.ActorLogMessages$$anon$1.applyOrElse(ActorLogMessages.scala:30)
	at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
	at org.apache.flink.runtime.jobmanager.JobManager.aroundReceive(JobManager.scala:91)
	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
	at akka.actor.ActorCell.invoke(ActorCell.scala:487)
	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254)
	at akka.dispatch.Mailbox.run(Mailbox.scala:221)
	at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.Exception: The data preparation for task 'CHAIN Join(Join at org.apache.flink.api.scala.UnfinishedJoinOperation.finish(joinDataSet.scala:224)) -> Map (Map at LowLevel.FlinkImplementation.metaOperator.SemiJoinMD$.apply(SemiJoinMD.scala:33)) -> Combine(Distinct at LowLevel.FlinkImplementation.metaOperator.SemiJoinMD$.apply(SemiJoinMD.scala:34))' , caused an error: Too few memory segments provided. Hash Join needs at least 33 memory segments.
	at org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:472)
	at org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
	at org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:217)
	at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.IllegalArgumentException: Too few memory segments provided. Hash Join needs at least 33 memory segments.
	at org.apache.flink.runtime.operators.hash.MutableHashTable.<init>(MutableHashTable.java:373)
	at org.apache.flink.runtime.operators.hash.MutableHashTable.<init>(MutableHashTable.java:359)
	at org.apache.flink.runtime.operators.hash.HashMatchIteratorBase.getHashJoin(HashMatchIteratorBase.java:48)
	at org.apache.flink.runtime.operators.hash.NonReusingBuildFirstHashMatchIterator.<init>(NonReusingBuildFirstHashMatchIterator.java:78)
	at org.apache.flink.runtime.operators.MatchDriver.prepare(MatchDriver.java:148)
	at org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:466)
	... 3 more




when this happens to solve it i need either to restart the computer or run it from the downloaded flink instance instead of the IDE

sometime the error is different: it says something like “you must start the memory manager with a positive value now is 0MB” but also this one of 33 memory segment is quite common 

it is quite annoying to restart everything and it happens quite often

do you have any idea why it happens?



thank you

Re: Memory exception

Posted by Michele Bertoni <mi...@mail.polimi.it>.
It just happened again and i solved with the -X parameter to the JVM

Thank you!


Il giorno 11/mag/2015, alle ore 21:50, Stephan Ewen <se...@apache.org>> ha scritto:

What you see is that the effect of extremely little memory arriving at the memory manager.

Given that you are saying that restarting the machine helps (or using a downloaded version of Flink), the reason is probably that you do not configure the JVM memory (in which case Java grabs a bit of the system's free memory). That is also why the downloaded installation works - it always starts Java with a fix memory size.

Try adding "-Xmx500m" or so to your "lauch" options in the IDE.



On Sun, May 10, 2015 at 9:19 PM, Michele Bertoni <mi...@mail.polimi.it>> wrote:
Hi,
thank you for your fast answer

next time it happens i will try

Michele



Il giorno 10/mag/2015, alle ore 19:45, Fabian Hueske <fh...@gmail.com>> ha scritto:

Hi Michele,

Flink manages its own memory using the MemoryManager. When a Flink worker (TaskManager) starts, the MemoryManager allocates a certain share of the JVM's free memory as MemorySegments: Default size of a MemorySegment is 32KB. When a Flink program is executed, the MemoryManager hands a certain number of MemorySegments to each processing operator such as HashJoin or Sort. These operators have a lower bound of MemorySegments which they need for operations, e.g., 33 MemorySegments for a HashJoin.

To solve your problem, you should try to give more memory to the Flink JVM. You can also configure a fixed amount of memory for the MemoryManager via the taskmanager.memory.size configuration parameter [1].

Cheers, Fabian

[1] http://ci.apache.org/projects/flink/flink-docs-master/setup/config.html#jobmanager-amp-taskmanager


2015-05-10 19:18 GMT+02:00 Michele Bertoni <mi...@mail.polimi.it>>:
Hi everybody,
I am running a flink instance on my IDE
sometimes (totally random) I start it i getting this error

Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
        at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$receiveWithLogMessages$1.applyOrElse(JobManager.scala:306)
        at scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
        at scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
        at scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
        at org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:37)
        at org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:30)
        at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118)
        at org.apache.flink.runtime.ActorLogMessages$$anon$1.applyOrElse(ActorLogMessages.scala:30)
        at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
        at org.apache.flink.runtime.jobmanager.JobManager.aroundReceive(JobManager.scala:91)
        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
        at akka.actor.ActorCell.invoke(ActorCell.scala:487)
        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254)
        at akka.dispatch.Mailbox.run(Mailbox.scala:221)
        at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.Exception: The data preparation for task 'CHAIN Join(Join at org.apache.flink.api.scala.UnfinishedJoinOperation.finish(joinDataSet.scala:224)) -> Map (Map at LowLevel.FlinkImplementation.metaOperator.SemiJoinMD$.apply(SemiJoinMD.scala:33)) -> Combine(Distinct at LowLevel.FlinkImplementation.metaOperator.SemiJoinMD$.apply(SemiJoinMD.scala:34))' , caused an error: Too few memory segments provided. Hash Join needs at least 33 memory segments.
        at org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:472)
        at org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
        at org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:217)
        at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.IllegalArgumentException: Too few memory segments provided. Hash Join needs at least 33 memory segments.
        at org.apache.flink.runtime.operators.hash.MutableHashTable.<init>(MutableHashTable.java:373)
        at org.apache.flink.runtime.operators.hash.MutableHashTable.<init>(MutableHashTable.java:359)
        at org.apache.flink.runtime.operators.hash.HashMatchIteratorBase.getHashJoin(HashMatchIteratorBase.java:48)
        at org.apache.flink.runtime.operators.hash.NonReusingBuildFirstHashMatchIterator.<init>(NonReusingBuildFirstHashMatchIterator.java:78)
        at org.apache.flink.runtime.operators.MatchDriver.prepare(MatchDriver.java:148)
        at org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:466)
        ... 3 more




when this happens to solve it i need either to restart the computer or run it from the downloaded flink instance instead of the IDE

sometime the error is different: it says something like “you must start the memory manager with a positive value now is 0MB” but also this one of 33 memory segment is quite common

it is quite annoying to restart everything and it happens quite often

do you have any idea why it happens?



thank you





Re: Memory exception

Posted by Stephan Ewen <se...@apache.org>.
What you see is that the effect of extremely little memory arriving at the
memory manager.

Given that you are saying that restarting the machine helps (or using a
downloaded version of Flink), the reason is probably that you do not
configure the JVM memory (in which case Java grabs a bit of the system's
free memory). That is also why the downloaded installation works - it
always starts Java with a fix memory size.

Try adding "-Xmx500m" or so to your "lauch" options in the IDE.



On Sun, May 10, 2015 at 9:19 PM, Michele Bertoni <
michele1.bertoni@mail.polimi.it> wrote:

>  Hi,
> thank you for your fast answer
>
>  next time it happens i will try
>
>  Michele
>
>
>
>  Il giorno 10/mag/2015, alle ore 19:45, Fabian Hueske <fh...@gmail.com>
> ha scritto:
>
>   Hi Michele,
>
>  Flink manages its own memory using the MemoryManager. When a Flink worker
> (TaskManager) starts, the MemoryManager allocates a certain share of the
> JVM's free memory as MemorySegments: Default size of a MemorySegment is
> 32KB. When a Flink program is executed, the MemoryManager hands a certain
> number of MemorySegments to each processing operator such as HashJoin or
> Sort. These operators have a lower bound of MemorySegments which they need
> for operations, e.g., 33 MemorySegments for a HashJoin.
>
>  To solve your problem, you should try to give more memory to the Flink
> JVM. You can also configure a fixed amount of memory for the MemoryManager
> via the taskmanager.memory.size configuration parameter [1].
>
>  Cheers, Fabian
>
> [1]
> http://ci.apache.org/projects/flink/flink-docs-master/setup/config.html#jobmanager-amp-taskmanager
>
>
> 2015-05-10 19:18 GMT+02:00 Michele Bertoni <
> michele1.bertoni@mail.polimi.it>:
>
>> Hi everybody,
>> I am running a flink instance on my IDE
>> sometimes (totally random) I start it i getting this error
>>
>> Exception in thread "main"
>> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
>>         at
>> org.apache.flink.runtime.jobmanager.JobManager$$anonfun$receiveWithLogMessages$1.applyOrElse(JobManager.scala:306)
>>         at
>> scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
>>         at
>> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
>>         at
>> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
>>         at
>> org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:37)
>>         at
>> org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:30)
>>         at
>> scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118)
>>         at
>> org.apache.flink.runtime.ActorLogMessages$$anon$1.applyOrElse(ActorLogMessages.scala:30)
>>         at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
>>         at
>> org.apache.flink.runtime.jobmanager.JobManager.aroundReceive(JobManager.scala:91)
>>         at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
>>         at akka.actor.ActorCell.invoke(ActorCell.scala:487)
>>         at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254)
>>         at akka.dispatch.Mailbox.run(Mailbox.scala:221)
>>         at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
>>         at
>> scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>>         at
>> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>>         at
>> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>>         at
>> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
>> Caused by: java.lang.Exception: The data preparation for task 'CHAIN
>> Join(Join at
>> org.apache.flink.api.scala.UnfinishedJoinOperation.finish(joinDataSet.scala:224))
>> -> Map (Map at
>> LowLevel.FlinkImplementation.metaOperator.SemiJoinMD$.apply(SemiJoinMD.scala:33))
>> -> Combine(Distinct at
>> LowLevel.FlinkImplementation.metaOperator.SemiJoinMD$.apply(SemiJoinMD.scala:34))'
>> , caused an error: Too few memory segments provided. Hash Join needs at
>> least 33 memory segments.
>>         at
>> org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:472)
>>         at
>> org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
>>         at
>> org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:217)
>>         at java.lang.Thread.run(Thread.java:722)
>> Caused by: java.lang.IllegalArgumentException: Too few memory segments
>> provided. Hash Join needs at least 33 memory segments.
>>         at
>> org.apache.flink.runtime.operators.hash.MutableHashTable.<init>(MutableHashTable.java:373)
>>         at
>> org.apache.flink.runtime.operators.hash.MutableHashTable.<init>(MutableHashTable.java:359)
>>         at
>> org.apache.flink.runtime.operators.hash.HashMatchIteratorBase.getHashJoin(HashMatchIteratorBase.java:48)
>>         at
>> org.apache.flink.runtime.operators.hash.NonReusingBuildFirstHashMatchIterator.<init>(NonReusingBuildFirstHashMatchIterator.java:78)
>>         at
>> org.apache.flink.runtime.operators.MatchDriver.prepare(MatchDriver.java:148)
>>         at
>> org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:466)
>>         ... 3 more
>>
>>
>>
>>
>> when this happens to solve it i need either to restart the computer or
>> run it from the downloaded flink instance instead of the IDE
>>
>> sometime the error is different: it says something like “you must start
>> the memory manager with a positive value now is 0MB” but also this one of
>> 33 memory segment is quite common
>>
>> it is quite annoying to restart everything and it happens quite often
>>
>> do you have any idea why it happens?
>>
>>
>>
>> thank you
>
>
>
>

Re: Memory exception

Posted by Michele Bertoni <mi...@mail.polimi.it>.
Hi,
thank you for your fast answer

next time it happens i will try

Michele


Il giorno 10/mag/2015, alle ore 19:45, Fabian Hueske <fh...@gmail.com>> ha scritto:

Hi Michele,

Flink manages its own memory using the MemoryManager. When a Flink worker (TaskManager) starts, the MemoryManager allocates a certain share of the JVM's free memory as MemorySegments: Default size of a MemorySegment is 32KB. When a Flink program is executed, the MemoryManager hands a certain number of MemorySegments to each processing operator such as HashJoin or Sort. These operators have a lower bound of MemorySegments which they need for operations, e.g., 33 MemorySegments for a HashJoin.

To solve your problem, you should try to give more memory to the Flink JVM. You can also configure a fixed amount of memory for the MemoryManager via the taskmanager.memory.size configuration parameter [1].

Cheers, Fabian

[1] http://ci.apache.org/projects/flink/flink-docs-master/setup/config.html#jobmanager-amp-taskmanager


2015-05-10 19:18 GMT+02:00 Michele Bertoni <mi...@mail.polimi.it>>:
Hi everybody,
I am running a flink instance on my IDE
sometimes (totally random) I start it i getting this error

Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
        at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$receiveWithLogMessages$1.applyOrElse(JobManager.scala:306)
        at scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
        at scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
        at scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
        at org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:37)
        at org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:30)
        at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118)
        at org.apache.flink.runtime.ActorLogMessages$$anon$1.applyOrElse(ActorLogMessages.scala:30)
        at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
        at org.apache.flink.runtime.jobmanager.JobManager.aroundReceive(JobManager.scala:91)
        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
        at akka.actor.ActorCell.invoke(ActorCell.scala:487)
        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254)
        at akka.dispatch.Mailbox.run(Mailbox.scala:221)
        at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.Exception: The data preparation for task 'CHAIN Join(Join at org.apache.flink.api.scala.UnfinishedJoinOperation.finish(joinDataSet.scala:224)) -> Map (Map at LowLevel.FlinkImplementation.metaOperator.SemiJoinMD$.apply(SemiJoinMD.scala:33)) -> Combine(Distinct at LowLevel.FlinkImplementation.metaOperator.SemiJoinMD$.apply(SemiJoinMD.scala:34))' , caused an error: Too few memory segments provided. Hash Join needs at least 33 memory segments.
        at org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:472)
        at org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
        at org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:217)
        at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.IllegalArgumentException: Too few memory segments provided. Hash Join needs at least 33 memory segments.
        at org.apache.flink.runtime.operators.hash.MutableHashTable.<init>(MutableHashTable.java:373)
        at org.apache.flink.runtime.operators.hash.MutableHashTable.<init>(MutableHashTable.java:359)
        at org.apache.flink.runtime.operators.hash.HashMatchIteratorBase.getHashJoin(HashMatchIteratorBase.java:48)
        at org.apache.flink.runtime.operators.hash.NonReusingBuildFirstHashMatchIterator.<init>(NonReusingBuildFirstHashMatchIterator.java:78)
        at org.apache.flink.runtime.operators.MatchDriver.prepare(MatchDriver.java:148)
        at org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:466)
        ... 3 more




when this happens to solve it i need either to restart the computer or run it from the downloaded flink instance instead of the IDE

sometime the error is different: it says something like “you must start the memory manager with a positive value now is 0MB” but also this one of 33 memory segment is quite common

it is quite annoying to restart everything and it happens quite often

do you have any idea why it happens?



thank you



Re: Memory exception

Posted by Fabian Hueske <fh...@gmail.com>.
Hi Michele,

Flink manages its own memory using the MemoryManager. When a Flink worker
(TaskManager) starts, the MemoryManager allocates a certain share of the
JVM's free memory as MemorySegments: Default size of a MemorySegment is
32KB. When a Flink program is executed, the MemoryManager hands a certain
number of MemorySegments to each processing operator such as HashJoin or
Sort. These operators have a lower bound of MemorySegments which they need
for operations, e.g., 33 MemorySegments for a HashJoin.

To solve your problem, you should try to give more memory to the Flink JVM.
You can also configure a fixed amount of memory for the MemoryManager via
the taskmanager.memory.size configuration parameter [1].

Cheers, Fabian

[1]
http://ci.apache.org/projects/flink/flink-docs-master/setup/config.html#jobmanager-amp-taskmanager


2015-05-10 19:18 GMT+02:00 Michele Bertoni <mi...@mail.polimi.it>
:

> Hi everybody,
> I am running a flink instance on my IDE
> sometimes (totally random) I start it i getting this error
>
> Exception in thread "main"
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
>         at
> org.apache.flink.runtime.jobmanager.JobManager$$anonfun$receiveWithLogMessages$1.applyOrElse(JobManager.scala:306)
>         at
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
>         at
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
>         at
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
>         at
> org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:37)
>         at
> org.apache.flink.runtime.ActorLogMessages$$anon$1.apply(ActorLogMessages.scala:30)
>         at
> scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118)
>         at
> org.apache.flink.runtime.ActorLogMessages$$anon$1.applyOrElse(ActorLogMessages.scala:30)
>         at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
>         at
> org.apache.flink.runtime.jobmanager.JobManager.aroundReceive(JobManager.scala:91)
>         at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
>         at akka.actor.ActorCell.invoke(ActorCell.scala:487)
>         at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254)
>         at akka.dispatch.Mailbox.run(Mailbox.scala:221)
>         at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
>         at
> scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>         at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>         at
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>         at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> Caused by: java.lang.Exception: The data preparation for task 'CHAIN
> Join(Join at
> org.apache.flink.api.scala.UnfinishedJoinOperation.finish(joinDataSet.scala:224))
> -> Map (Map at
> LowLevel.FlinkImplementation.metaOperator.SemiJoinMD$.apply(SemiJoinMD.scala:33))
> -> Combine(Distinct at
> LowLevel.FlinkImplementation.metaOperator.SemiJoinMD$.apply(SemiJoinMD.scala:34))'
> , caused an error: Too few memory segments provided. Hash Join needs at
> least 33 memory segments.
>         at
> org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:472)
>         at
> org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
>         at
> org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:217)
>         at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.IllegalArgumentException: Too few memory segments
> provided. Hash Join needs at least 33 memory segments.
>         at
> org.apache.flink.runtime.operators.hash.MutableHashTable.<init>(MutableHashTable.java:373)
>         at
> org.apache.flink.runtime.operators.hash.MutableHashTable.<init>(MutableHashTable.java:359)
>         at
> org.apache.flink.runtime.operators.hash.HashMatchIteratorBase.getHashJoin(HashMatchIteratorBase.java:48)
>         at
> org.apache.flink.runtime.operators.hash.NonReusingBuildFirstHashMatchIterator.<init>(NonReusingBuildFirstHashMatchIterator.java:78)
>         at
> org.apache.flink.runtime.operators.MatchDriver.prepare(MatchDriver.java:148)
>         at
> org.apache.flink.runtime.operators.RegularPactTask.run(RegularPactTask.java:466)
>         ... 3 more
>
>
>
>
> when this happens to solve it i need either to restart the computer or run
> it from the downloaded flink instance instead of the IDE
>
> sometime the error is different: it says something like “you must start
> the memory manager with a positive value now is 0MB” but also this one of
> 33 memory segment is quite common
>
> it is quite annoying to restart everything and it happens quite often
>
> do you have any idea why it happens?
>
>
>
> thank you