You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by James Chris Beaver <be...@algonquincollege.com> on 2011/12/14 21:09:46 UTC
Question about disk optimization in ATS.
Hello Folks,
I am looking for information in regards to the best way to optimize disk in ATS. I understand that the application uses raw disks however what is the best way to lay out those disks? Is it best to have more than one cache disk? I am inclined to put up more than one hardware raid5 volume and then format them as raw disks for ATS to make use of. Is there a "rule of thumb" when it comes to disk sizing? What block size is best (large of small)? For example if you have to large of a disk(s) will that slow down ATS as it must spend time keeping track of very large caches?
Any guidance is greatly appreciated.
Regards
--
Chris Beaver
UNIX Systems Administrator, ITS - Algonquin College
1385 Woodroffe Ave
Ottawa, ON K2G 1V8
email: beaverj@algonquincollege.com<ma...@algonquincollege.com>
phone: (613) 727-4723 x 5385
fax: (613) 727-7719
Re: Question about disk optimization in ATS.
Posted by John Plevyak <jp...@gmail.com>.
>
> Basically, for every 8KB of disk cache, one directory entry is created.
> And each directory entry consumes memory (10 bytes if I recall). So, if you
> average object sizes are much larger than e.g. 8KB, it's worthwhile to tune
> this setting. Likewise if your average objects are much smaller than 8k.
> There's also some overhead when ATS "syncs" this directory to disk (the
> larger the directory, the more disk I/O to sync it to disk).
>
Yes, the main overhead is that large disks are going to spend more time
syncing the directory. You probably don't need tons of multi-terabyte
disks unless you are doing large documents (e.g. video) in which case you
could change that configuration to decrease the directory size. 6x146GB
just as separate disks should have good performance in most scenarios
(Zipf-ish distribution and working set comfortably within the 800GB).
>
> Don't ask me why the default is 8000 and not 8192, John might know :).
No reason.
Re: Question about disk optimization in ATS.
Posted by Leif Hedstrom <zw...@apache.org>.
On 12/14/11 2:18 PM, James Chris Beaver wrote:
> Is there an algorithm for sizing? Is it possible to have a disk that is too large? Is it possible to have a too many disks?
Yes, you are limited by RAM. The more disk, the more RAM you will consume.
There's also a configuration here that will help with RAM consumption:
CONFIG proxy.config.cache.min_average_object_size INT 8000
Basically, for every 8KB of disk cache, one directory entry is created. And
each directory entry consumes memory (10 bytes if I recall). So, if you
average object sizes are much larger than e.g. 8KB, it's worthwhile to tune
this setting. Likewise if your average objects are much smaller than 8k.
There's also some overhead when ATS "syncs" this directory to disk (the
larger the directory, the more disk I/O to sync it to disk).
Don't ask me why the default is 8000 and not 8192, John might know :).
-- leif
RE: Question about disk optimization in ATS.
Posted by James Chris Beaver <be...@algonquincollege.com>.
Is there an algorithm for sizing? Is it possible to have a disk that is too large? Is it possible to have a too many disks?
Regards
Chris
-----Original Message-----
From: Igor Galić [mailto:i.galic@brainsware.org]
Sent: Wednesday, December 14, 2011 4:14 PM
To: users@trafficserver.apache.org
Subject: Re: Question about disk optimization in ATS.
----- Original Message -----
>
>
>
> Thanks for the insight John.
>
>
>
> When you say no RAID of any kind – I was going to format 2 large
> hardware raid 5 sets for redundancy. The raid 5 sets would be
> comprised of 3 * 146GB SCSI drives (as there are two raid 5 sets there
> are 6 disks in total). Are you saying that it would be best if I just
> presented 6 (un-raided) disks to be formatted as raw disks?
Yes.
> Kind Regards
>
>
>
> Chris
>
>
>
>
> From: John Plevyak [mailto:jplevyak@gmail.com]
> Sent: Wednesday, December 14, 2011 4:04 PM
> To: users@trafficserver.apache.org
> Subject: Re: Question about disk optimization in ATS.
>
>
>
>
>
>
>
> Under heavy forward proxy load, ATS tends to be seek limited which
> favors lots of disks. As ATS has its own recovery mechanisms it is
> best *not* to use raid5 or any raid, but just a bunch of raw disks
> listed in the storage.config.
>
>
>
>
>
> ATS needs to have small block sizes, so don't use raid or stripe with
> a volume manager as this will degrade performance.
>
>
>
>
>
> If you use a disk which has large blocks (3GB disk) then you may need
> to set the flag to force ATS to use a larger internal blocks size
> which results in some waste of disk space.
>
>
>
>
>
> If you are reverse proxy and have a small working set and a good size
> RAM cache you may not be disk limited, in which case you have more
> flexibility. Even in this case I would
>
>
> still suggest not using raid or striping.
>
>
>
>
>
> john
>
>
>
>
> On Wed, Dec 14, 2011 at 12:09 PM, James Chris Beaver <
> beaverj@algonquincollege.com > wrote:
>
>
>
>
>
> Hello Folks,
>
>
>
> I am looking for information in regards to the best way to optimize
> disk in ATS. I understand that the application uses raw disks however
> what is the best way to lay out those disks? Is it best to have more
> than one cache disk? I am inclined to put up more than one hardware
> raid5 volume and then format them as raw disks for ATS to make use of.
> Is there a “rule of thumb” when it comes to disk sizing? What block
> size is best (large of small)? For example if you have to large of a
> disk(s) will that slow down ATS as it must spend time keeping track of
> very large caches?
>
>
>
> Any guidance is greatly appreciated.
>
>
>
> Regards
>
>
>
> --
>
> Chris Beaver
>
> UNIX Systems Administrator, ITS - Algonquin College
>
> 1385 Woodroffe Ave
>
> Ottawa, ON K2G 1V8
>
> email: beaverj@algonquincollege.com
>
> phone: (613) 727-4723 x 5385
>
> fax: (613) 727-7719
>
>
>
>
>
>
>
>
>
>
>
>
--
Igor Galić
Tel: +43 (0) 664 886 22 883
Mail: i.galic@brainsware.org
URL: http://brainsware.org/
GPG: 6880 4155 74BD FD7C B515 2EA5 4B1D 9E08 A097 C9AE
--
Re: Question about disk optimization in ATS.
Posted by Igor Galić <i....@brainsware.org>.
----- Original Message -----
>
>
>
> Thanks for the insight John.
>
>
>
> When you say no RAID of any kind – I was going to format 2 large
> hardware raid 5 sets for redundancy. The raid 5 sets would be
> comprised of 3 * 146GB SCSI drives (as there are two raid 5 sets
> there are 6 disks in total). Are you saying that it would be best if
> I just presented 6 (un-raided) disks to be formatted as raw disks?
Yes.
> Kind Regards
>
>
>
> Chris
>
>
>
>
> From: John Plevyak [mailto:jplevyak@gmail.com]
> Sent: Wednesday, December 14, 2011 4:04 PM
> To: users@trafficserver.apache.org
> Subject: Re: Question about disk optimization in ATS.
>
>
>
>
>
>
>
> Under heavy forward proxy load, ATS tends to be seek limited which
> favors lots of disks. As ATS has its own recovery mechanisms it is
> best *not* to use raid5 or any raid, but just a bunch of raw disks
> listed in the storage.config.
>
>
>
>
>
> ATS needs to have small block sizes, so don't use raid or stripe with
> a volume manager as this will degrade performance.
>
>
>
>
>
> If you use a disk which has large blocks (3GB disk) then you may need
> to set the flag to force ATS to use a larger internal blocks size
> which results in some waste of disk space.
>
>
>
>
>
> If you are reverse proxy and have a small working set and a good size
> RAM cache you may not be disk limited, in which case you have more
> flexibility. Even in this case I would
>
>
> still suggest not using raid or striping.
>
>
>
>
>
> john
>
>
>
>
> On Wed, Dec 14, 2011 at 12:09 PM, James Chris Beaver <
> beaverj@algonquincollege.com > wrote:
>
>
>
>
>
> Hello Folks,
>
>
>
> I am looking for information in regards to the best way to optimize
> disk in ATS. I understand that the application uses raw disks
> however what is the best way to lay out those disks? Is it best to
> have more than one cache disk? I am inclined to put up more than one
> hardware raid5 volume and then format them as raw disks for ATS to
> make use of. Is there a “rule of thumb” when it comes to disk
> sizing? What block size is best (large of small)? For example if you
> have to large of a disk(s) will that slow down ATS as it must spend
> time keeping track of very large caches?
>
>
>
> Any guidance is greatly appreciated.
>
>
>
> Regards
>
>
>
> --
>
> Chris Beaver
>
> UNIX Systems Administrator, ITS - Algonquin College
>
> 1385 Woodroffe Ave
>
> Ottawa, ON K2G 1V8
>
> email: beaverj@algonquincollege.com
>
> phone: (613) 727-4723 x 5385
>
> fax: (613) 727-7719
>
>
>
>
>
>
>
>
>
>
>
>
--
Igor Galić
Tel: +43 (0) 664 886 22 883
Mail: i.galic@brainsware.org
URL: http://brainsware.org/
GPG: 6880 4155 74BD FD7C B515 2EA5 4B1D 9E08 A097 C9AE
RE: Question about disk optimization in ATS.
Posted by James Chris Beaver <be...@algonquincollege.com>.
Thanks for the insight John.
When you say no RAID of any kind - I was going to format 2 large hardware raid 5 sets for redundancy. The raid 5 sets would be comprised of 3 * 146GB SCSI drives (as there are two raid 5 sets there are 6 disks in total). Are you saying that it would be best if I just presented 6 (un-raided) disks to be formatted as raw disks?
Kind Regards
Chris
From: John Plevyak [mailto:jplevyak@gmail.com]
Sent: Wednesday, December 14, 2011 4:04 PM
To: users@trafficserver.apache.org
Subject: Re: Question about disk optimization in ATS.
Under heavy forward proxy load, ATS tends to be seek limited which favors lots of disks. As ATS has its own recovery mechanisms it is best *not* to use raid5 or any raid, but just a bunch of raw disks listed in the storage.config.
ATS needs to have small block sizes, so don't use raid or stripe with a volume manager as this will degrade performance.
If you use a disk which has large blocks (3GB disk) then you may need to set the flag to force ATS to use a larger internal blocks size which results in some waste of disk space.
If you are reverse proxy and have a small working set and a good size RAM cache you may not be disk limited, in which case you have more flexibility. Even in this case I would
still suggest not using raid or striping.
john
On Wed, Dec 14, 2011 at 12:09 PM, James Chris Beaver <be...@algonquincollege.com>> wrote:
Hello Folks,
I am looking for information in regards to the best way to optimize disk in ATS. I understand that the application uses raw disks however what is the best way to lay out those disks? Is it best to have more than one cache disk? I am inclined to put up more than one hardware raid5 volume and then format them as raw disks for ATS to make use of. Is there a "rule of thumb" when it comes to disk sizing? What block size is best (large of small)? For example if you have to large of a disk(s) will that slow down ATS as it must spend time keeping track of very large caches?
Any guidance is greatly appreciated.
Regards
--
Chris Beaver
UNIX Systems Administrator, ITS - Algonquin College
1385 Woodroffe Ave
Ottawa, ON K2G 1V8
email: beaverj@algonquincollege.com<ma...@algonquincollege.com>
phone: (613) 727-4723 x 5385<tel:%28613%29%20727-4723%20x%205385>
fax: (613) 727-7719<tel:%28613%29%20727-7719>
________________________________
Re: Question about disk optimization in ATS.
Posted by John Plevyak <jp...@gmail.com>.
Under heavy forward proxy load, ATS tends to be seek limited which favors
lots of disks. As ATS has its own recovery mechanisms it is best *not* to
use raid5 or any raid, but just a bunch of raw disks listed in the
storage.config.
ATS needs to have small block sizes, so don't use raid or stripe with a
volume manager as this will degrade performance.
If you use a disk which has large blocks (3GB disk) then you may need to
set the flag to force ATS to use a larger internal blocks size which
results in some waste of disk space.
If you are reverse proxy and have a small working set and a good size RAM
cache you may not be disk limited, in which case you have more flexibility.
Even in this case I would
still suggest not using raid or striping.
john
On Wed, Dec 14, 2011 at 12:09 PM, James Chris Beaver <
beaverj@algonquincollege.com> wrote:
> ** **
>
> Hello Folks,****
>
> ** **
>
> I am looking for information in regards to the best way to optimize disk
> in ATS. I understand that the application uses raw disks however what is
> the best way to lay out those disks? Is it best to have more than one
> cache disk? I am inclined to put up more than one hardware raid5 volume
> and then format them as raw disks for ATS to make use of. Is there a “rule
> of thumb” when it comes to disk sizing? What block size is best (large of
> small)? For example if you have to large of a disk(s) will that slow down
> ATS as it must spend time keeping track of very large caches?****
>
> ** **
>
> Any guidance is greatly appreciated.****
>
> ** **
>
> Regards****
>
> ** **
>
> -- ****
>
> Chris Beaver****
>
> UNIX Systems Administrator, ITS - Algonquin College****
>
> 1385 Woodroffe Ave****
>
> Ottawa, ON K2G 1V8****
>
> email: beaverj@algonquincollege.com****
>
> phone: (613) 727-4723 x 5385****
>
> fax: (613) 727-7719****
>
> ** **
>