You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Eduardo Costa Alfaia <e....@unibs.it> on 2014/04/08 00:12:44 UTC
Driver Out of Memory
Hi Guys,
I would like understanding why the Driver's RAM goes down, Does the
processing occur only in the workers?
Thanks
##### Start Tests
computer1(Worker/Source Stream)
23:57:18 up 12:03, 1 user, load average: 0.03, 0.31, 0.44
total used free shared buffers cached
Mem: 3945 1084 2860 0 44 827
-/+ buffers/cache: 212 3732
Swap: 0 0 0
computer8 (Driver/Master)
23:57:18 up 11:53, 5 users, load average: 0.43, 1.19, 1.31
total used free shared buffers cached
Mem: 5897 4430 1466 0 384 2662
-/+ buffers/cache: 1382 4514
Swap: 0 0 0
computer10(Worker/Source Stream)
23:57:18 up 12:02, 1 user, load average: 0.55, 1.34, 0.98
total used free shared buffers cached
Mem: 5897 564 5332 0 18 358
-/+ buffers/cache: 187 5709
Swap: 0 0 0
computer11(Worker/Source Stream)
23:57:18 up 12:02, 1 user, load average: 0.07, 0.19, 0.29
total used free shared buffers cached
Mem: 3945 603 3342 0 54 355
-/+ buffers/cache: 193 3751
Swap: 0 0 0
#### After 2 Minutes
computer1
00:06:41 up 12:12, 1 user, load average: 3.11, 1.32, 0.73
total used free shared buffers cached
Mem: 3945 2950 994 0 46 1095
-/+ buffers/cache: 1808 2136
Swap: 0 0 0
computer8(Driver/Master)
00:06:41 up 12:02, 5 users, load average: 1.16, 0.71, 0.96
total used free shared buffers cached
Mem: 5897 5191 705 0 385 2792
-/+ buffers/cache: 2014 3882
Swap: 0 0 0
computer10
00:06:41 up 12:11, 1 user, load average: 2.02, 1.07, 0.89
total used free shared buffers cached
Mem: 5897 2567 3329 0 21 647
-/+ buffers/cache: 1898 3998
Swap: 0 0 0
computer11
00:06:42 up 12:12, 1 user, load average: 3.96, 1.83, 0.88
total used free shared buffers cached
Mem: 3945 3542 402 0 57 1099
-/+ buffers/cache: 2385 1559
Swap: 0 0 0
--
Informativa sulla Privacy: http://www.unibs.it/node/8155
Re: Driver Out of Memory
Posted by Aaron Davidson <il...@gmail.com>.
The driver does not do processing (for the most part), but it does do
scheduling and block management, so it can keep around a significant amount
of metadata about all the stored RDD and broadcast blocks, as well as
statistics about prior executions. Much of this is bounded in some way,
though, so it is unusual for the driver to use too much memory. If you
could get a heap dump (using jmap) of the driver when it's using a lot of
memory, that would likely be very informative about what exactly is piling
up.
On Mon, Apr 7, 2014 at 3:12 PM, Eduardo Costa Alfaia <e.costaalfaia@unibs.it
> wrote:
> Hi Guys,
>
> I would like understanding why the Driver's RAM goes down, Does the
> processing occur only in the workers?
> Thanks
> ##### Start Tests
> computer1(Worker/Source Stream)
> 23:57:18 up 12:03, 1 user, load average: 0.03, 0.31, 0.44
> total used free shared buffers cached
> Mem: 3945 1084 2860 0 44 827
> -/+ buffers/cache: 212 3732
> Swap: 0 0 0
> computer8 (Driver/Master)
> 23:57:18 up 11:53, 5 users, load average: 0.43, 1.19, 1.31
> total used free shared buffers cached
> Mem: 5897 4430 1466 0 384 2662
> -/+ buffers/cache: 1382 4514
> Swap: 0 0 0
> computer10(Worker/Source Stream)
> 23:57:18 up 12:02, 1 user, load average: 0.55, 1.34, 0.98
> total used free shared buffers cached
> Mem: 5897 564 5332 0 18 358
> -/+ buffers/cache: 187 5709
> Swap: 0 0 0
> computer11(Worker/Source Stream)
> 23:57:18 up 12:02, 1 user, load average: 0.07, 0.19, 0.29
> total used free shared buffers cached
> Mem: 3945 603 3342 0 54 355
> -/+ buffers/cache: 193 3751
> Swap: 0 0 0
>
> #### After 2 Minutes
>
> computer1
> 00:06:41 up 12:12, 1 user, load average: 3.11, 1.32, 0.73
> total used free shared buffers cached
> Mem: 3945 2950 994 0 46 1095
> -/+ buffers/cache: 1808 2136
> Swap: 0 0 0
> computer8(Driver/Master)
> 00:06:41 up 12:02, 5 users, load average: 1.16, 0.71, 0.96
> total used free shared buffers cached
> Mem: 5897 5191 705 0 385 2792
> -/+ buffers/cache: 2014 3882
> Swap: 0 0 0
> computer10
> 00:06:41 up 12:11, 1 user, load average: 2.02, 1.07, 0.89
> total used free shared buffers cached
> Mem: 5897 2567 3329 0 21 647
> -/+ buffers/cache: 1898 3998
> Swap: 0 0 0
> computer11
> 00:06:42 up 12:12, 1 user, load average: 3.96, 1.83, 0.88
> total used free shared buffers cached
> Mem: 3945 3542 402 0 57 1099
> -/+ buffers/cache: 2385 1559
> Swap: 0 0 0
>
>
> Informativa sulla Privacy: http://www.unibs.it/node/8155