GC limit exceed

[marco rocchi <ro...@studenti.uniroma1.it>]

I have a spark code that works well over a sample of data in local mode,
but when I pass the same code on a cluster with the entire dataset I
receive GC limited exceed error.
In that section is possible to submit the code and have some hints in order
to solve my problem?
Thanks a lot for the attention

Marco

Re: GC limit exceed

[Daniel van der Ende <da...@gmail.com>]

Hi Marco,

What kind of scheduler are you using on your cluster? Yarn?

Also, are you running in client mode or cluster mode on the cluster?

Daniel

On Wed, Jan 18, 2017 at 3:22 PM, marco rocchi <
rocchi.1407763@studenti.uniroma1.it> wrote:

> I have a spark code that works well over a sample of data in local mode,
> but when I pass the same code on a cluster with the entire dataset I
> receive GC limited exceed error.
> In that section is possible to submit the code and have some hints in
> order to solve my problem?
> Thanks a lot for the attention
>
> Marco
>



-- 
Daniel