You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Steve Malenfant <sm...@gmail.com> on 2016/03/27 04:29:22 UTC
ATS Disk load distribution
Hi,
Our servers have 24 cache disks. On each of them, I'm seing an unusual
pattern where the 14th cache disk (/dev/sdn here) has half the tps and
Blk_read of the others while running a simple iostat.
The OS drive is /dev/sdy.
Is there something obvious that could explain this? Anybody else seeing
similar behaviors?
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sdy 8.50 0.59 412.30 1182292 826749224
sdb 17.96 5574.30 763.26 11177773390 1530520872
sda 17.27 5427.78 753.80 10883976079 1511537961
sdc 16.76 5426.23 752.01 10880866108 1507966007
sdf 17.40 5415.19 754.99 10858730317 1513930486
sdd 17.42 5415.10 750.04 10858548347 1504011560
sdg 17.43 5482.40 757.33 10993500408 1518615277
sde 16.49 5265.87 740.41 10559295722 1484695034
sdh 17.38 5492.09 758.66 11012923059 1521284992
sdj 18.73 5849.73 787.82 11730078650 1579772953
sdk 16.85 5319.37 742.29 10666574619 1488471133
sdl 17.53 5608.22 766.14 11245803256 1536281866
sdn 8.58 2555.06 517.71 5123501157 1038125596
sdm 17.69 5525.47 757.92 11079862986 1519798735
sdo 15.75 5039.02 726.77 10104411633 1457343705
sdi 18.33 5684.66 782.37 11399081848 1568839983
sdq 17.49 5596.34 760.76 11221981240 1525498204
sdp 16.31 5143.23 734.07 10313388901 1471978836
sdr 16.96 5393.52 755.80 10815269418 1515562524
sdt 16.47 5293.78 740.87 10615265266 1485622464
sds 15.99 5201.40 732.21 10430031258 1468246477
sdv 17.92 5684.24 779.27 11398240262 1562614587
sdu 17.26 5504.14 760.25 11037086102 1524475490
sdx 17.74 5580.22 762.95 11189655745 1529890237
sdw 17.17 5308.27 744.56 10644334189 1493013627
Steve
Re: ATS Disk load distribution
Posted by Steve Malenfant <sm...@gmail.com>.
We already have this specified. This specific /dev/sdn device is just
receiving half the load compared to others across all our caches.
On Sun, Mar 27, 2016 at 2:01 PM, Schmerge, John (Contractor) <
John_Schmerge@cable.comcast.com> wrote:
> Hi Steve,
>
>
> Try adding a "volume=X" to each line of your storage.config. There's an
> old issue with ATS when you have a single volume and replace a drive, in
> which it (silently) does not use it because because it cannot figure out
> which volume the drive belongs to. Explicitly assigning the volume in
> storage.config sidesteps this condition.
>
>
> -John
>
>
> ------------------------------
> *From:* Steve Malenfant <sm...@gmail.com>
> *Sent:* Saturday, March 26, 2016 10:29 PM
> *To:* users@trafficserver.apache.org
> *Subject:* ATS Disk load distribution
>
> Hi,
>
> Our servers have 24 cache disks. On each of them, I'm seing an unusual
> pattern where the 14th cache disk (/dev/sdn here) has half the tps and
> Blk_read of the others while running a simple iostat.
>
> The OS drive is /dev/sdy.
>
> Is there something obvious that could explain this? Anybody else seeing
> similar behaviors?
>
> Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
> sdy 8.50 0.59 412.30 1182292 826749224
> sdb 17.96 5574.30 763.26 11177773390 1530520872
> sda 17.27 5427.78 753.80 10883976079 1511537961
> sdc 16.76 5426.23 752.01 10880866108 1507966007
> sdf 17.40 5415.19 754.99 10858730317 1513930486
> sdd 17.42 5415.10 750.04 10858548347 1504011560
> sdg 17.43 5482.40 757.33 10993500408 1518615277
> sde 16.49 5265.87 740.41 10559295722 1484695034
> sdh 17.38 5492.09 758.66 11012923059 1521284992
> sdj 18.73 5849.73 787.82 11730078650 1579772953
> sdk 16.85 5319.37 742.29 10666574619 1488471133
> sdl 17.53 5608.22 766.14 11245803256 1536281866
> sdn 8.58 2555.06 517.71 5123501157 1038125596
> sdm 17.69 5525.47 757.92 11079862986 1519798735
> sdo 15.75 5039.02 726.77 10104411633 1457343705
> sdi 18.33 5684.66 782.37 11399081848 1568839983
> sdq 17.49 5596.34 760.76 11221981240 1525498204
> sdp 16.31 5143.23 734.07 10313388901 1471978836
> sdr 16.96 5393.52 755.80 10815269418 1515562524
> sdt 16.47 5293.78 740.87 10615265266 1485622464
> sds 15.99 5201.40 732.21 10430031258 1468246477
> sdv 17.92 5684.24 779.27 11398240262 1562614587
> sdu 17.26 5504.14 760.25 11037086102 1524475490
> sdx 17.74 5580.22 762.95 11189655745 1529890237
> sdw 17.17 5308.27 744.56 10644334189 1493013627
>
> Steve
>
Re: ATS Disk load distribution
Posted by "Schmerge, John (Contractor)" <Jo...@cable.comcast.com>.
Hi Steve,
Try adding a "volume=X" to each line of your storage.config. There's an old issue with ATS when you have a single volume and replace a drive, in which it (silently) does not use it because because it cannot figure out which volume the drive belongs to. Explicitly assigning the volume in storage.config sidesteps this condition.
-John
________________________________
From: Steve Malenfant <sm...@gmail.com>
Sent: Saturday, March 26, 2016 10:29 PM
To: users@trafficserver.apache.org
Subject: ATS Disk load distribution
Hi,
Our servers have 24 cache disks. On each of them, I'm seing an unusual pattern where the 14th cache disk (/dev/sdn here) has half the tps and Blk_read of the others while running a simple iostat.
The OS drive is /dev/sdy.
Is there something obvious that could explain this? Anybody else seeing similar behaviors?
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sdy 8.50 0.59 412.30 1182292 826749224
sdb 17.96 5574.30 763.26 11177773390 1530520872
sda 17.27 5427.78 753.80 10883976079 1511537961
sdc 16.76 5426.23 752.01 10880866108 1507966007
sdf 17.40 5415.19 754.99 10858730317 1513930486
sdd 17.42 5415.10 750.04 10858548347 1504011560
sdg 17.43 5482.40 757.33 10993500408 1518615277
sde 16.49 5265.87 740.41 10559295722 1484695034
sdh 17.38 5492.09 758.66 11012923059 1521284992
sdj 18.73 5849.73 787.82 11730078650 1579772953
sdk 16.85 5319.37 742.29 10666574619 1488471133
sdl 17.53 5608.22 766.14 11245803256 1536281866
sdn 8.58 2555.06 517.71 5123501157 1038125596
sdm 17.69 5525.47 757.92 11079862986 1519798735
sdo 15.75 5039.02 726.77 10104411633 1457343705
sdi 18.33 5684.66 782.37 11399081848 1568839983
sdq 17.49 5596.34 760.76 11221981240 1525498204
sdp 16.31 5143.23 734.07 10313388901 1471978836
sdr 16.96 5393.52 755.80 10815269418 1515562524
sdt 16.47 5293.78 740.87 10615265266 1485622464
sds 15.99 5201.40 732.21 10430031258 1468246477
sdv 17.92 5684.24 779.27 11398240262 1562614587
sdu 17.26 5504.14 760.25 11037086102 1524475490
sdx 17.74 5580.22 762.95 11189655745 1529890237
sdw 17.17 5308.27 744.56 10644334189 1493013627
Steve