You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by ZHAOCHUNBO, , 赵春波, , ZH...@kingsoft.com on 2016/04/16 04:26:08 UTC

答复: ram_cache for big objects

Hi Sudheer,

Thank you very much for you suggestion.

But I think ramdisk is not suitable for my case. There are many other files in the same domain, the storage requirement is larger than my ram storage.
Only several of them are hot files, may I have a way to make ram_cache or linux pagecache work?

Thanks in advance,
Thomas.


________________________________________
发件人: Sudheer Vinukonda [sudheerv@yahoo-inc.com]
发送时间: 2016年4月15日 23:47
收件人: users@trafficserver.apache.org
主题: Re: ram_cache for big objects

One approach is to use a ramdisk (e.g configure tmpfs ).


On Friday, April 15, 2016 12:33 AM, ZHAOCHUNBO [赵春波] <ZH...@kingsoft.com> wrote:


HI all,

I have met one problem:

There is one big file(600MB) and it is very hot, it is larger than ` ram_cache_cutoff ` setup:
CONFIG proxy.config.cache.ram_cache_cutoff INT 20M


I have noticed that the `rKB/s` on one disk is very high, and the disk usage is keeping nearly 100%.

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sde               0.00   124.00  196.33    2.33 47210.00   505.33   480.36    35.17  182.13   5.02  99.73


It seems linux pagecache does NOT work when using ATS, and ATS ram_cache does not work either, am I correct?
What is the suggestion value of `proxy.config.cache.ram_cache.size` and `proxy.config.cache.ram_cache_cutoff` ?

How should I handle this case?


Thanks in advance.

---
Best Regards,
Thomas

Re: 答复: ram_cache for big objects

Posted by Sudheer Vinukonda <su...@yahoo-inc.com>.
You can map a volume to a specific disk (in your case ramdisk) in storage.config.

http://trafficserver.readthedocs.org/en/latest/admin-guide/files/storage.config.en.html



Thanks,

Sudheer









On Monday, April 18, 2016 5:32 AM, ZHAOCHUNBO [赵春波] <ZH...@kingsoft.com> wrote:



 
HI Sudheer,
 
Acording to the document of “volume.conf”, it seems I can not assign ramdisk to one specified volume:
 
http://trafficserver.readthedocs.org/en/latest/admin-guide/files/volume.config.en.html
Each volume is striped across several disks to achieve parallel I/O. For example: if there are four disks, then a 1-GB volume will have 256 MB on each disk (assuming each disk has enough free space available).
 
So do you mean any other plugins?
 
 
Thanks,
Thomas.
 
 
 
---------------------------------------------------------------------------------
赵春波Thomas   CDN架构师
邮箱:  zhaochunbo@kingsoft.com
手机：(+86)18511746860
地址：北京市海淀区小营西路33号金山软件大厦3F
邮编：100085
---------------------------------------------------------------------------------
 
From:Sudheer Vinukonda [mailto:sudheerv@yahoo-inc.com] 
Sent: Saturday, April 16, 2016 10:58 AM
To: ZHAOCHUNBO [赵春波]; users@trafficserver.apache.org
Subject: Re: 答复: ram_cache for big objects
 
I haven't had to rely on page cache with ATS (unsure it'd work reliably, given ATS's appetite for memory - ram cache + non ram cache).
 
But, you can configure multiple cache partitions with the required files mapped to ramdisk (using volumes) and the rest mapped to regular disk. Cache volumes allow mapping based on host name (hosting.config). If the domain is not unique, you can modify the cache key to add a special domain name for the files needed and store them in the volumes that are tied to the ramdisk.
 
 
http://trafficserver.readthedocs.org/en/latest/admin-guide/storage/index.en.html#partitioning-the-cache

http://trafficserver.readthedocs.org/en/latest/admin-guide/files/hosting.config.en.html#std:configfile-hosting.config


On Friday, April 15, 2016, 7:27 PM, ZHAOCHUNBO [赵春波] <ZH...@kingsoft.com> wrote:
Hi Sudheer,
>
>Thank you very much for you suggestion.
>
>But I think ramdisk is not suitable for my case. There are many other files in the same domain, the storage requirement is larger than my ram storage.
>Only several of them are hot files, may I have a way to make ram_cache or linux pagecache work?
>
>Thanks in advance,
>Thomas.
>
>
>________________________________________
>
>发件人: Sudheer Vinukonda [sudheerv@yahoo-inc.com]
>发送时间: 2016年4月15日23:47
>收件人: users@trafficserver.apache.org
>主题: Re: ram_cache for big objects
>
>One approach is to use a ramdisk (e.g configure tmpfs ).
>
>
>On Friday, April 15, 2016 12:33 AM, ZHAOCHUNBO [赵春波] <ZH...@kingsoft.com> wrote:
>
>
>HI all,
>
>I have met one problem:
>
>There is one big file(600MB) and it is very hot, it is larger than ` ram_cache_cutoff ` setup:
>CONFIG proxy.config.cache.ram_cache_cutoff INT 20M
>
>
>I have noticed that the `rKB/s` on one disk is very high, and the disk usage is keeping nearly 100%.
>
>Device:        rrqm/s  wrqm/s    r/s    w/s    rkB/s    wkB/s avgrq-sz avgqu-sz  await  svctm  %util
>sde              0.00  124.00  196.33    2.33 47210.00  505.33  480.36    35.17  182.13  5.02  99.73
>
>
>It seems linux pagecache does NOT work when using ATS, and ATS ram_cache does not work either, am I correct?
>What is the suggestion value of `proxy.config.cache.ram_cache.size` and `proxy.config.cache.ram_cache_cutoff` ?
>
>How should I handle this case?
>
>
>Thanks in advance.
>
>---
>Best Regards,
>Thomas
>
>
>

RE: 答复: ram_cache for big objects

Posted by ZHAOCHUNBO, , 赵春波, , ZH...@kingsoft.com.
HI Sudheer,

Acording to the document of “volume.conf”, it seems I can not assign ramdisk to one specified volume:

http://trafficserver.readthedocs.org/en/latest/admin-guide/files/volume.config.en.html
Each volume is striped across several disks to achieve parallel I/O. For example: if there are four disks, then a 1-GB volume will have 256 MB on each disk (assuming each disk has enough free space available).

So do you mean any other plugins?


Thanks,
Thomas.



---------------------------------------------------------------------------------
[cid:image001.png@01D199B0.D4412CF0]
赵春波 Thomas   CDN架构师
邮箱 :  zhaochunbo@kingsoft.com<ma...@kingsoft.com>
手机：(+86)18511746860
地址：北京市海淀区小营西路33号金山软件大厦3F
邮编：100085
---------------------------------------------------------------------------------

From: Sudheer Vinukonda [mailto:sudheerv@yahoo-inc.com]
Sent: Saturday, April 16, 2016 10:58 AM
To: ZHAOCHUNBO [赵春波]; users@trafficserver.apache.org
Subject: Re: 答复: ram_cache for big objects

I haven't had to rely on page cache with ATS (unsure it'd work reliably, given ATS's appetite for memory - ram cache + non ram cache).

But, you can configure multiple cache partitions with the required files mapped to ramdisk (using volumes) and the rest mapped to regular disk. Cache volumes allow mapping based on host name (hosting.config). If the domain is not unique, you can modify the cache key to add a special domain name for the files needed and store them in the volumes that are tied to the ramdisk.


http://trafficserver.readthedocs.org/en/latest/admin-guide/storage/index.en.html#partitioning-the-cache

http://trafficserver.readthedocs.org/en/latest/admin-guide/files/hosting.config.en.html#std:configfile-hosting.config


On Friday, April 15, 2016, 7:27 PM, ZHAOCHUNBO [赵春波] <ZH...@kingsoft.com>> wrote:
Hi Sudheer,

Thank you very much for you suggestion.

But I think ramdisk is not suitable for my case. There are many other files in the same domain, the storage requirement is larger than my ram storage.
Only several of them are hot files, may I have a way to make ram_cache or linux pagecache work?

Thanks in advance,
Thomas.


________________________________________

发件人: Sudheer Vinukonda [sudheerv@yahoo-inc.com<javascript:return>]
发送时间: 2016年4月15日 23:47
收件人: users@trafficserver.apache.org<javascript:return>
主题: Re: ram_cache for big objects

One approach is to use a ramdisk (e.g configure tmpfs ).


On Friday, April 15, 2016 12:33 AM, ZHAOCHUNBO [赵春波] <ZHAOCHUNBO@kingsoft.com<javascript:return>> wrote:


HI all,

I have met one problem:

There is one big file(600MB) and it is very hot, it is larger than ` ram_cache_cutoff ` setup:
CONFIG proxy.config.cache.ram_cache_cutoff INT 20M


I have noticed that the `rKB/s` on one disk is very high, and the disk usage is keeping nearly 100%.

Device:        rrqm/s  wrqm/s    r/s    w/s    rkB/s    wkB/s avgrq-sz avgqu-sz  await  svctm  %util
sde              0.00  124.00  196.33    2.33 47210.00  505.33  480.36    35.17  182.13  5.02  99.73


It seems linux pagecache does NOT work when using ATS, and ATS ram_cache does not work either, am I correct?
What is the suggestion value of `proxy.config.cache.ram_cache.size` and `proxy.config.cache.ram_cache_cutoff` ?

How should I handle this case?


Thanks in advance.

---
Best Regards,
Thomas

Re: 答复: ram_cache for big objects

Posted by Sudheer Vinukonda <su...@yahoo-inc.com>.
 blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px #715FFA solid !important; padding-left:1ex !important; background-color:white !important; } I haven't had to rely on page cache with ATS (unsure it'd work reliably, given ATS's appetite for memory - ram cache + non ram cache).
But, you can configure multiple cache partitions with the required files mapped to ramdisk (using volumes) and the rest mapped to regular disk. Cache volumes allow mapping based on host name (hosting.config). If the domain is not unique, you can modify the cache key to add a special domain name for the files needed and store them in the volumes that are tied to the ramdisk.

http://trafficserver.readthedocs.org/en/latest/admin-guide/storage/index.en.html#partitioning-the-cache
http://trafficserver.readthedocs.org/en/latest/admin-guide/files/hosting.config.en.html#std:configfile-hosting.config



On Friday, April 15, 2016, 7:27 PM, ZHAOCHUNBO [赵春波] <ZH...@kingsoft.com> wrote:

Hi Sudheer,

Thank you very much for you suggestion.

But I think ramdisk is not suitable for my case. There are many other files in the same domain, the storage requirement is larger than my ram storage.
Only several of them are hot files, may I have a way to make ram_cache or linux pagecache work?

Thanks in advance,
Thomas.


________________________________________
发件人: Sudheer Vinukonda [sudheerv@yahoo-inc.com]
发送时间: 2016年4月15日 23:47
收件人: users@trafficserver.apache.org
主题: Re: ram_cache for big objects

One approach is to use a ramdisk (e.g configure tmpfs ).


On Friday, April 15, 2016 12:33 AM, ZHAOCHUNBO [赵春波] <ZH...@kingsoft.com> wrote:


HI all,

I have met one problem:

There is one big file(600MB) and it is very hot, it is larger than ` ram_cache_cutoff ` setup:
CONFIG proxy.config.cache.ram_cache_cutoff INT 20M


I have noticed that the `rKB/s` on one disk is very high, and the disk usage is keeping nearly 100%.

Device:        rrqm/s  wrqm/s    r/s    w/s    rkB/s    wkB/s avgrq-sz avgqu-sz  await  svctm  %util
sde              0.00  124.00  196.33    2.33 47210.00  505.33  480.36    35.17  182.13  5.02  99.73


It seems linux pagecache does NOT work when using ATS, and ATS ram_cache does not work either, am I correct?
What is the suggestion value of `proxy.config.cache.ram_cache.size` and `proxy.config.cache.ram_cache_cutoff` ?

How should I handle this case?


Thanks in advance.

---
Best Regards,
Thomas