You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by qingyang li <li...@gmail.com> on 2014/03/07 09:09:55 UTC
how to get size of rdd in memery
dear community, can anyone tell me : how to get size of rdd in memery ?
thanks.
Re: how to get size of rdd in memery
Posted by qingyang li <li...@gmail.com>.
addtion :
1. i have run LOAD DATA INPATH '/user/root/input/test.txt' into table b; in
shark. i think this will create rdd in memery, right?
2. when i run "free -g" , the result show somethings has been stored into
memery. the file is almost 4g.
[root@bigdata001 spark-0.9.0-incubating-bin-hadoop2]# free -g
total used free shared buffers cached
Mem: 15 6 8 0 0 4
-/+ buffers/cache: 2 13
Swap: 7 0 7
2014-03-07 16:51 GMT+08:00 qingyang li <li...@gmail.com>:
> in that page, it is empty , it does not show anything.
> Here is the picture.
>
>
>
>
> 2014-03-07 16:14 GMT+08:00 Mayur Rustagi <ma...@gmail.com>:
>
> http://<Master URL>:4040/storage/
>>
>> Mayur Rustagi
>> Ph: +1 (760) 203 3257
>> http://www.sigmoidanalytics.com
>> @mayur_rustagi <https://twitter.com/mayur_rustagi>
>>
>>
>>
>> On Fri, Mar 7, 2014 at 12:09 AM, qingyang li <li...@gmail.com>wrote:
>>
>>> dear community, can anyone tell me : how to get size of rdd in memery ?
>>> thanks.
>>>
>>
>>
>
Re: how to get size of rdd in memery
Posted by qingyang li <li...@gmail.com>.
addtion :
1. i have run LOAD DATA INPATH '/user/root/input/test.txt' into table b; in
shark. i think this will create rdd in memery, right?
2. when i run "free -g" , the result show somethings has been stored into
memery. the file is almost 4g.
[root@bigdata001 spark-0.9.0-incubating-bin-hadoop2]# free -g
total used free shared buffers cached
Mem: 15 6 8 0 0 4
-/+ buffers/cache: 2 13
Swap: 7 0 7
2014-03-07 16:51 GMT+08:00 qingyang li <li...@gmail.com>:
> in that page, it is empty , it does not show anything.
> Here is the picture.
>
>
>
>
> 2014-03-07 16:14 GMT+08:00 Mayur Rustagi <ma...@gmail.com>:
>
> http://<Master URL>:4040/storage/
>>
>> Mayur Rustagi
>> Ph: +1 (760) 203 3257
>> http://www.sigmoidanalytics.com
>> @mayur_rustagi <https://twitter.com/mayur_rustagi>
>>
>>
>>
>> On Fri, Mar 7, 2014 at 12:09 AM, qingyang li <li...@gmail.com>wrote:
>>
>>> dear community, can anyone tell me : how to get size of rdd in memery ?
>>> thanks.
>>>
>>
>>
>
Re: how to get size of rdd in memery
Posted by qingyang li <li...@gmail.com>.
in that page, it is empty , it does not show anything.
Here is the picture.
2014-03-07 16:14 GMT+08:00 Mayur Rustagi <ma...@gmail.com>:
> http://<Master URL>:4040/storage/
>
> Mayur Rustagi
> Ph: +1 (760) 203 3257
> http://www.sigmoidanalytics.com
> @mayur_rustagi <https://twitter.com/mayur_rustagi>
>
>
>
> On Fri, Mar 7, 2014 at 12:09 AM, qingyang li <li...@gmail.com>wrote:
>
>> dear community, can anyone tell me : how to get size of rdd in memery ?
>> thanks.
>>
>
>
Re: how to get size of rdd in memery
Posted by Mayur Rustagi <ma...@gmail.com>.
http://<Master URL>:4040/storage/
Mayur Rustagi
Ph: +1 (760) 203 3257
http://www.sigmoidanalytics.com
@mayur_rustagi <https://twitter.com/mayur_rustagi>
On Fri, Mar 7, 2014 at 12:09 AM, qingyang li <li...@gmail.com>wrote:
> dear community, can anyone tell me : how to get size of rdd in memery ?
> thanks.
>