You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by Ben <be...@meh.net.nz> on 2014/11/03 12:54:40 UTC
Re: Sample server configuration for 10 Gbps traffic
On Mon, Nov 03, 2014 at 01:59:11PM +0200, Ergin Ozekes wrote:
> Hi guys;
>
> I need sample server configuration for 10 Gbps traffic. Which configuration
> will be recommended for 10 Gpbs traffic load?
Even a default config can push 10 gigabit of traffic over a local network to a single
user fine.
If you want to have lots of users then you need fast storage etc. It all depends how
much is cached vs new data etc. So many open questions, that you're going to have to
be more specific.
Ben.
Re: Sample server configuration for 10 Gbps traffic
Posted by Ergin Ozekes <er...@gmail.com>.
Hi Leif;
>
> > Hi Ben;
> >
> > It will daily store approximately 2.8 Tb data. In our network 20.000
> users
> > are exist. The ratio for cached vs new data is approximately %30. Also
> that
> > system generates 30 Gb log.
> > In according to these information what is your recommendations?
>
> You expect to “write” 2.8TB of new data every day? How much of that is
> active typically? It’s difficult to say without knowing usage patterns, but
> if cost is a factor (and it usually is :), I’d probably stay with
> rotational drives over SSD for a very large data set, and use some of that
> money for more RAM. I’d probably go at least 64GB of RAM on such a system.
>
>
Probably, we will use 512 GB RAM on production system. Cache ram size will
be 400GB.
> As for ATS configs goes, the defaults would probably be just fine with
> that setup, at least initially, except for manually setting up the RAM
> cache. The one thing you can look at which can reduce memory usage on very
> large disk caches is proxy.config.cache.min_average_object_size. Increasing
> that will reduce memory consumption, but also reduce the total number of
> objects that the cache can hold. So, if you have a lot of very large
> objects, increasing this makes sense.
>
> For 10Gig NICs, there might be interesting things to do around IRQ
> balancing, and ring buffers and that sort of stuff.
I've done something the increasing the ring buffer by using ethtool like
(ethtool -G ethx rx 4078) and used the large buffer offload ( ethtool -K
lro on). Btw I've increased the kernel buffer size and ATS in and out r/w
buffer size.
> I should get our rocket scientist devops guys here to write something up.
> Ben and John, are you reading this? :).
>
> Cheers,
>
> — Leif
>
>
Best Regards,
Ergin
Re: Sample server configuration for 10 Gbps traffic
Posted by Leif Hedstrom <zw...@apache.org>.
> On Nov 3, 2014, at 5:27 AM, Ergin Ozekes <er...@gmail.com> wrote:
>
> Hi Ben;
>
> It will daily store approximately 2.8 Tb data. In our network 20.000 users
> are exist. The ratio for cached vs new data is approximately %30. Also that
> system generates 30 Gb log.
> In according to these information what is your recommendations?
You expect to “write” 2.8TB of new data every day? How much of that is active typically? It’s difficult to say without knowing usage patterns, but if cost is a factor (and it usually is :), I’d probably stay with rotational drives over SSD for a very large data set, and use some of that money for more RAM. I’d probably go at least 64GB of RAM on such a system.
As for ATS configs goes, the defaults would probably be just fine with that setup, at least initially, except for manually setting up the RAM cache. The one thing you can look at which can reduce memory usage on very large disk caches is proxy.config.cache.min_average_object_size. Increasing that will reduce memory consumption, but also reduce the total number of objects that the cache can hold. So, if you have a lot of very large objects, increasing this makes sense.
For 10Gig NICs, there might be interesting things to do around IRQ balancing, and ring buffers and that sort of stuff. I should get our rocket scientist devops guys here to write something up. Ben and John, are you reading this? :).
Cheers,
— Leif
Re: Sample server configuration for 10 Gbps traffic
Posted by Ergin Ozekes <er...@gmail.com>.
I've already send the subscribe email to dev and users list. I will try one
more time.
On Mon, Nov 3, 2014 at 5:24 PM, Leif Hedstrom <le...@ogre.com> wrote:
>
> > On Nov 3, 2014, at 8:12 AM, Ergin Ozekes <er...@gmail.com> wrote:
> >
> > Hi All;
> >
> > Do you have any recommendations?
> >
> > Best Regards,
>
>
> Beware, the Cc: line includes a “subscribe” email address, so unless you
> intend to subscribe to that mailing list, don’t reply all :).
>
> — leif
>
> > Ergin
> >
> > On Mon, Nov 3, 2014 at 2:27 PM, Ergin Ozekes <er...@gmail.com>
> wrote:
> >
> >> Hi Ben;
> >>
> >> It will daily store approximately 2.8 Tb data. In our network 20.000
> users
> >> are exist. The ratio for cached vs new data is approximately %30. Also
> that
> >> system generates 30 Gb log.
> >> In according to these information what is your recommendations?
> >>
> >> Best Regards,
> >> Ergin Ozekes
> >> Turksat
> >>
> >> On Mon, Nov 3, 2014 at 1:54 PM, Ben <be...@meh.net.nz> wrote:
> >>
> >>> On Mon, Nov 03, 2014 at 01:59:11PM +0200, Ergin Ozekes wrote:
> >>>> Hi guys;
> >>>>
> >>>> I need sample server configuration for 10 Gbps traffic. Which
> >>> configuration
> >>>> will be recommended for 10 Gpbs traffic load?
> >>>
> >>> Even a default config can push 10 gigabit of traffic over a local
> network
> >>> to a single
> >>> user fine.
> >>>
> >>> If you want to have lots of users then you need fast storage etc. It
> all
> >>> depends how
> >>> much is cached vs new data etc. So many open questions, that you're
> >>> going to have to
> >>> be more specific.
> >>>
> >>> Ben.
> >>>
> >>
> >>
>
>
Re: Sample server configuration for 10 Gbps traffic
Posted by Leif Hedstrom <le...@ogre.com>.
> On Nov 3, 2014, at 8:12 AM, Ergin Ozekes <er...@gmail.com> wrote:
>
> Hi All;
>
> Do you have any recommendations?
>
> Best Regards,
Beware, the Cc: line includes a “subscribe” email address, so unless you intend to subscribe to that mailing list, don’t reply all :).
— leif
> Ergin
>
> On Mon, Nov 3, 2014 at 2:27 PM, Ergin Ozekes <er...@gmail.com> wrote:
>
>> Hi Ben;
>>
>> It will daily store approximately 2.8 Tb data. In our network 20.000 users
>> are exist. The ratio for cached vs new data is approximately %30. Also that
>> system generates 30 Gb log.
>> In according to these information what is your recommendations?
>>
>> Best Regards,
>> Ergin Ozekes
>> Turksat
>>
>> On Mon, Nov 3, 2014 at 1:54 PM, Ben <be...@meh.net.nz> wrote:
>>
>>> On Mon, Nov 03, 2014 at 01:59:11PM +0200, Ergin Ozekes wrote:
>>>> Hi guys;
>>>>
>>>> I need sample server configuration for 10 Gbps traffic. Which
>>> configuration
>>>> will be recommended for 10 Gpbs traffic load?
>>>
>>> Even a default config can push 10 gigabit of traffic over a local network
>>> to a single
>>> user fine.
>>>
>>> If you want to have lots of users then you need fast storage etc. It all
>>> depends how
>>> much is cached vs new data etc. So many open questions, that you're
>>> going to have to
>>> be more specific.
>>>
>>> Ben.
>>>
>>
>>
Re: Sample server configuration for 10 Gbps traffic
Posted by Ergin Ozekes <er...@gmail.com>.
Hi All;
Do you have any recommendations?
Best Regards,
Ergin
On Mon, Nov 3, 2014 at 2:27 PM, Ergin Ozekes <er...@gmail.com> wrote:
> Hi Ben;
>
> It will daily store approximately 2.8 Tb data. In our network 20.000 users
> are exist. The ratio for cached vs new data is approximately %30. Also that
> system generates 30 Gb log.
> In according to these information what is your recommendations?
>
> Best Regards,
> Ergin Ozekes
> Turksat
>
> On Mon, Nov 3, 2014 at 1:54 PM, Ben <be...@meh.net.nz> wrote:
>
>> On Mon, Nov 03, 2014 at 01:59:11PM +0200, Ergin Ozekes wrote:
>> > Hi guys;
>> >
>> > I need sample server configuration for 10 Gbps traffic. Which
>> configuration
>> > will be recommended for 10 Gpbs traffic load?
>>
>> Even a default config can push 10 gigabit of traffic over a local network
>> to a single
>> user fine.
>>
>> If you want to have lots of users then you need fast storage etc. It all
>> depends how
>> much is cached vs new data etc. So many open questions, that you're
>> going to have to
>> be more specific.
>>
>> Ben.
>>
>
>
Re: Sample server configuration for 10 Gbps traffic
Posted by Ergin Ozekes <er...@gmail.com>.
Hi Ben;
It will daily store approximately 2.8 Tb data. In our network 20.000 users
are exist. The ratio for cached vs new data is approximately %30. Also that
system generates 30 Gb log.
In according to these information what is your recommendations?
Best Regards,
Ergin Ozekes
Turksat
On Mon, Nov 3, 2014 at 1:54 PM, Ben <be...@meh.net.nz> wrote:
> On Mon, Nov 03, 2014 at 01:59:11PM +0200, Ergin Ozekes wrote:
> > Hi guys;
> >
> > I need sample server configuration for 10 Gbps traffic. Which
> configuration
> > will be recommended for 10 Gpbs traffic load?
>
> Even a default config can push 10 gigabit of traffic over a local network
> to a single
> user fine.
>
> If you want to have lots of users then you need fast storage etc. It all
> depends how
> much is cached vs new data etc. So many open questions, that you're going
> to have to
> be more specific.
>
> Ben.
>