You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Karthick S <ka...@gmail.com> on 2013/11/08 10:31:54 UTC
ATS - Performance Related Questions - High
Hi Team,
I would like to know few performance related information. We have capacity
planning meeting for setting up a new cluster with AT. So your input will
be a great value add in our capacity planning.
Questions:
1) YTS(Yahoo Traffic Server) has some performance optimization which
utilizes the RAID-10 level for effective cache lookup. Does this ATS also
holds the similar type of feature which will gel well with RAID 10 level.
If not we will go ahead with RAID 0 to save more disk space for larger
cache. Currently we have 300GB cache on each ATS nodes.
2) How does ATS scales with respect to CPU , Memory, Disk? We have our
current production running with YTS and 16GB ram. Will ATS scale its
performance If I am getting 24GB ram ? How ATS will scale with respect to
hardware. Can you please give some insight about it.
3) What is the QPS of current ATS?
PS: Please treat this email as high priority, all your inputs will be
valuable to me.
Regards,
Karthick
Re: ATS - Performance Related Questions - High
Posted by Leif Hedstrom <zw...@apache.org>.
On Nov 8, 2013, at 2:07 AM, Igor Galić <i....@brainsware.org> wrote:
>
>
>> holds the similar type of feature which will gel well with RAID 10 level.
>> If not we will go ahead with RAID 0 to save more disk space for larger
>> cache. Currently we have 300GB cache on each ATS nodes.
>
> Given that Yahoo has switched from using YTS to using ATS, it's
> probably safe to say that they consider the performance to be,
> at least, sufficient.
The cache does not change between YTS and ATS in this regard. Use JBOD unless you have specific needs for e.g. RAID10; One such use case I can think of is when you have very, very large files, then JBOD is undesirable.
> https://trafficserver.readthedocs.org/en/latest/admin/configuring-cache.en.html
>
>> 2) How does ATS scales with respect to CPU , Memory, Disk? We have our
>
> CPU: when compiling with --enable-hwloc (highly recommended) much,
> much better.
Yes, our defaults are better than they used to be. You are still strongly encouraged to tune your settings according to your hardware and application.
>
> Memory: We've recently increased the RAM cache backing by an order
> of magnitude. IIRC, that means that for your 300 G disk(s) ATS will
> now use 3G RAM.
Note that this is just a default. You can (and almost always should) tune this as per your HW and app specs.
>
> Disk: we spawn one thread for each disk spindle (all the more reason
> to not use RAID and directly expose disks to ATS).
Correction: 8 I/O threads / disk *by default*. If you use RAID, you should tune this accordingly as well. The optimal setting here also depends on the actual disk, so tune it.
>
>> current production running with YTS and 16GB ram. Will ATS scale its
>> performance If I am getting 24GB ram ? How ATS will scale with respect to
>> hardware. Can you please give some insight about it.
>
>
> Generally we try to auto-scale with the underlying hardware. But
> you can always make adjustments to fit your load and working set.
+1.
As for RAM, it’s used in three primary ways:
1) In-memory indices for the disk storage. The bigger the disk, the more memory. Look at proxy.config.cache.min_average_object_size. The math is (ballpark)
10 * (disk size / proxy.config.cache.min_average_object_size)
This is allocated up front, and also gets synced to disk periodically; during sync time, more memory can be used for “snapshots”.
2) RAM cache as autoscaled or explicitly configured with proxy.config.cache.ram_cache.size. This is allocated as necessary (so you will see memory growth until it hit this cap).
3) Connections. SSL connections consumes more than regular ones. I haven’t kept up with exactly we are with this, it used to be 32KB / 64KB minimum for an inactive connection, and potentially much more for an active one (buffers).
>
>> 3) What is the QPS of current ATS?
Depends on your application; i.e. impossible to tell, benchmark ATS vs Varnish vs Squid for your app.
— leif
Re: ATS - Performance Related Questions - High
Posted by Igor Galić <i....@brainsware.org>.
----- Original Message -----
> Hi Team,
>
> I would like to know few performance related information. We have capacity
> planning meeting for setting up a new cluster with AT. So your input will
> be a great value add in our capacity planning.
>
> Questions:
> 1) YTS(Yahoo Traffic Server) has some performance optimization which
> utilizes the RAID-10 level for effective cache lookup. Does this ATS also
> holds the similar type of feature which will gel well with RAID 10 level.
> If not we will go ahead with RAID 0 to save more disk space for larger
> cache. Currently we have 300GB cache on each ATS nodes.
Given that Yahoo has switched from using YTS to using ATS, it's
probably safe to say that they consider the performance to be,
at least, sufficient.
Generally, the recommendation is to not use RAID at all. If you are
running on bare-metal, just give all the disks you are intending to
use as cache directly to ATS. It can handle the balancing itself.
you can find more detail and pointers here:
https://trafficserver.readthedocs.org/en/latest/admin/configuring-cache.en.html
> 2) How does ATS scales with respect to CPU , Memory, Disk? We have our
CPU: when compiling with --enable-hwloc (highly recommended) much,
much better.
Memory: We've recently increased the RAM cache backing by an order
of magnitude. IIRC, that means that for your 300 G disk(s) ATS will
now use 3G RAM.
Disk: we spawn one thread for each disk spindle (all the more reason
to not use RAID and directly expose disks to ATS).
> current production running with YTS and 16GB ram. Will ATS scale its
> performance If I am getting 24GB ram ? How ATS will scale with respect to
> hardware. Can you please give some insight about it.
Generally we try to auto-scale with the underlying hardware. But
you can always make adjustments to fit your load and working set.
> 3) What is the QPS of current ATS?
>
> PS: Please treat this email as high priority, all your inputs will be
> valuable to me.
This is a user and developer mailing list of an open source project.
Many people here are volunteers, and, while others are paid to work
on ATS, they do that for paying customers. If you wish to become
their highly prioritized paying customer, you can look select from
one of
https://trafficserver.apache.org/assistance.html
> Regards,
> Karthick
>
So long,
++ i
Igor Galić
Tel: +43 (0) 664 886 22 883
Mail: i.galic@brainsware.org
URL: http://brainsware.org/
GPG: 8716 7A9F 989B ABD5 100F 4008 F266 55D6 2998 1641
Re: ATS - Performance Related Questions - High
Posted by Jean Baptiste Favre <we...@jbfavre.org>.
On 09/11/2013 15:38, Reindl Harald wrote:
>
> Am 09.11.2013 15:25, schrieb Jean Baptiste Favre:
>> On 09/11/2013 09:13, Omid Kosari wrote:
>>> Bryan Call wrote
>>>> Don't compile your own version of ATS and use the packages on dist.
>>>
>>> Unfortunately dists are a bit lazy to implement latest versions . How we can
>>> use 4.x on ubuntu ?
>>
>> You can backport it :)
>>
>> But, since Ubuntu is based on Debian and Debian does not provides 4.x
>> yet, you'll have to adapt packaging from an earlier version. Using 3.3.2
>> version available in Debian experimental is a good start.
>>
>> Another solution is to build ATS from source. But then, you'll loose
>> packaging advantages
>
> says who?
>
> people which seriously maintain servers could/should learn how to make
> packages and that is pretty sure doable on Debian systems too
If you build any software from source and install it outside packaging
system, then, yes, you loose packaging advantages.
You'll get harder times to upgrade it and/or deploy it.
That said, I agree any serious admin should learn how to package
software for his platform. And it's definitely doable on Debian :)
This gives you the opportunity to enhance package to make it suits
*your* needs. for exemple, integrating fixes which are not yet released
(TS-1821 for Linux native AIO, just as an example :) ), and/or enable
experimental modules.
That said, I guess I'm one of them, running Debian Wheezy :)
# ls -1d trafficserver*
trafficserver
trafficserver_4.0.2-1~bbc72+1_amd64.changes
trafficserver_4.0.2-1~bbc72+1_amd64.deb
trafficserver_4.0.2-1~bbc72+1.debian.tar.gz
trafficserver_4.0.2-1~bbc72+1.dsc
trafficserver-4.0.2.tar.bz2
trafficserver-dev_4.0.2-1~bbc72+1_amd64.deb
Regards,
Jean-Baptiste
Re: ATS - Performance Related Questions - High
Posted by Reindl Harald <h....@thelounge.net>.
Am 09.11.2013 15:25, schrieb Jean Baptiste Favre:
> On 09/11/2013 09:13, Omid Kosari wrote:
>> Bryan Call wrote
>>> Don't compile your own version of ATS and use the packages on dist.
>>
>> Unfortunately dists are a bit lazy to implement latest versions . How we can
>> use 4.x on ubuntu ?
>
> You can backport it :)
>
> But, since Ubuntu is based on Debian and Debian does not provides 4.x
> yet, you'll have to adapt packaging from an earlier version. Using 3.3.2
> version available in Debian experimental is a good start.
>
> Another solution is to build ATS from source. But then, you'll loose
> packaging advantages
says who?
people which seriously maintain servers could/should learn how to make
packages and that is pretty sure doable on Debian systems too
-rw-r----- 1 builduser builduser 2,4K 2013-10-30 21:36 x264-latest.spec
[builduser@buildserver:/rpmbuild/SPECS]$ ls -lha *.spec | wc -l
82
[root@proxy:~]$ rpm -q trafficserver
trafficserver-4.0.2-2.fc18.20131027.rh.x86_64
________________________________________________
[builduser@buildserver:/rpmbuild/SPECS]$ cat trafficserver.spec
%global debug_package %{nil}
Summary: Apache Trafficserver
Name: trafficserver
Version: 4.0.2
Release: 2%{?dist}
License: ASL 2.0
Group: System Environment/Daemons
Source0: http://www.apache.org/dist/%{name}/%{name}-%{version}.tar.bz2
Source1: %{name}.service
Source2: %{name}.tmpfilesd
Patch1: trafficserver-404-not-found.patch
URL: http://trafficserver.apache.org/index.html
BuildRequires: autoconf
BuildRequires: automake
BuildRequires: expat-devel
BuildRequires: gcc-c++
BuildRequires: hwloc-devel
BuildRequires: libtool
BuildRequires: openssl-devel
BuildRequires: pcre-devel
BuildRequires: tcl-devel
BuildRequires: xz-devel
BuildRequires: zlib-devel
Requires: systemd
%description
Apache Traffic Server is a fast, scalable and extensible HTTP/1.1 compliant caching proxy server
%package devel
Summary: Apache Traffic Server development libraries and header files
Group: Development/Libraries
Requires: %{name} = %{version}-%{release}
%description devel
%package manpages
Summary: Trafficserver manuals
Group: Applications/System
%description manpages
%package plugins
Summary: Trafficserver plugins
Group: Applications/System
%description plugins
%prep
%setup -q
%patch1 -p1
%build
export CFLAGS="%{optflags} -fPIC -fPIE -fstack-protector-all -Wno-deprecated-declarations -Wno-error=unused-result"
export CXXFLAGS="%{optflags} -fPIC -fPIE -fstack-protector-all -Wno-deprecated-declarations -Wno-error=unused-result"
export LDFLAGS="-Wl,-z,now -Wl,-z,relro,-z,noexecstack -pie"
export SH_LDFLAGS="-Wl,-z,now -Wl,-z,relro,-z,noexecstack -pie"
./configure --enable-layout=Gentoo \
--libdir=%{_libdir}/%{name} \
--with-tcl=%{_libdir} \
--with-jemalloc \
--with-user=ats \
--with-group=ats \
--with-pic \
--enable-shared \
--enable-reclaimable-freelist \
--enable-linux-native-aio \
--enable-hwloc \
--disable-diags \
--disable-debug \
--disable-static
sed -i -e 's|^hardcode_libdir_flag_spec=.*|hardcode_libdir_flag_spec=""|g' -e
's|^runpath_var=LD_RUN_PATH|runpath_var=DIE_RPATH_DIE|g' libtool
%{__make} %{?_smp_mflags}
%install
make install-strip DESTDIR=%{buildroot}
mkdir -p %{buildroot}%{_mandir}/man1
mkdir -p %{buildroot}/run/%{name}
cp doc/man/*.1 %{buildroot}%{_mandir}/man1/
mv %{buildroot}%{_mandir}/man1/enable.1 %{buildroot}%{_mandir}/man1/ts-enable.1
mv %{buildroot}%{_mandir}/man1/disable.1 %{buildroot}%{_mandir}/man1/ts-disable.1
mv %{buildroot}%{_mandir}/man1/exit.1 %{buildroot}%{_mandir}/man1/ts-exit.1
install -D -m 0644 -p %{SOURCE1} %{buildroot}%{_unitdir}/%{name}.service
install -D -m 0644 -p %{SOURCE2} %{buildroot}%{_prefix}/lib/tmpfiles.d/%{name}.conf
rm -rf %{buildroot}%{_docdir}/%{name}
rm -rf %{buildroot}%{_libdir}/perl5/
rm -f %{buildroot}%{_libdir}/%{name}/*.a
rm -f %{buildroot}%{_libdir}/%{name}/*.la
rm -f %{buildroot}%{_libdir}/%{name}/plugins/*.la
rm -f %{buildroot}%{_datarootdir}/perl5/Apache/TS.pm
rm -f %{buildroot}%{_datarootdir}/perl5/Apache/TS/AdminClient.pm
rm -f %{buildroot}%{_datarootdir}/perl5/Apache/TS/Config.pm
rm -f %{buildroot}%{_datarootdir}/perl5/Apache/TS/Config/Records.pm
rm -f %{buildroot}%{_bindir}/tspush
rm -f %{buildroot}%{_bindir}/traffic_shell
rm -f %{buildroot}%{_bindir}/traffic_sac
rm -f %{buildroot}%{_bindir}/trafficserver
rm -f %{buildroot}%{_bindir}/tstop
strip -s %{buildroot}%{_bindir}/traffic_cop
strip -s %{buildroot}%{_bindir}/traffic_line
strip -s %{buildroot}%{_bindir}/traffic_logcat
strip -s %{buildroot}%{_bindir}/traffic_logstats
strip -s %{buildroot}%{_bindir}/traffic_server
strip -s %{buildroot}%{_libdir}/%{name}/*.so*
strip -s %{buildroot}%{_libdir}/%{name}/plugins/*.so
mkdir -p %{buildroot}%{_sysconfdir}/ld.so.conf.d
mkdir -p %{buildroot}%{_sysconfdir}/%{name}/internal
mkdir -p %{buildroot}%{_sysconfdir}/%{name}/snapshots
mkdir -p %{buildroot}%{_sysconfdir}/%{name}/ssl
mkdir -p %{buildroot}%{_docdir}/%{name}/etc/body_factory/default
chmod 0770 %{buildroot}%{_sysconfdir}/%{name}/internal
chmod 0770 %{buildroot}%{_sysconfdir}/%{name}/snapshots
chmod 0750 %{buildroot}%{_sysconfdir}/%{name}/ssl
chmod 0750 %{buildroot}%{_sysconfdir}/%{name}/body_factory
echo "%{_libdir}/%{name}" > %{buildroot}%{_sysconfdir}/ld.so.conf.d/%{name}-%{_arch}.conf
mv %{buildroot}%{_sysconfdir}/%{name}/body_factory/default/* %{buildroot}%{_docdir}/%{name}/etc/body_factory/default/
mv %{buildroot}%{_sysconfdir}/%{name}/*.config %{buildroot}%{_docdir}/%{name}/etc/
%post
%{_sbindir}/ldconfig
%{_bindir}/systemctl --system daemon-reload 2> /dev/null > /dev/null
chown root:ats %{_sysconfdir}/%{name}/body_factory 2> /dev/null > /dev/null
chown root:ats %{_sysconfdir}/%{name}/ssl 2> /dev/null > /dev/null
chown ats:ats %{_sysconfdir}/%{name}/internal 2> /dev/null > /dev/null
chown ats:ats %{_sysconfdir}/%{name}/snapshots 2> /dev/null > /dev/null
exit 0
%pre
getent group ats >/dev/null || groupadd -r ats -g 176 &>/dev/null
getent passwd ats >/dev/null || useradd -r -u 176 -g ats -d / -s %{_sbindir}/nologin -c "Apache Traffic Server" ats
&>/dev/null
exit 0
%files
%dir %{_libdir}/%{name}
%dir %{_sysconfdir}/%{name}
%config(noreplace) %{_sysconfdir}/%{name}/*
%{_bindir}/traffic*
%{_libdir}/%{name}/*.so.*
%{_unitdir}/%{name}.service
%{_prefix}/lib/tmpfiles.d/%{name}.conf
%attr(0770, ats, ats) %dir %{_var}/log/%{name}
%attr(0770, ats, ats) %dir /run/%{name}
%attr(0770, ats, ats) %dir %{_var}/cache/%{name}
%attr(0644, root, root) %{_sysconfdir}/ld.so.conf.d/*
%files devel
%{_bindir}/tsxs
%dir %{_includedir}/ts
%{_includedir}/ts/*
%{_libdir}/%{name}/*.so
%files manpages
%{_mandir}/man1/*
%{_mandir}/man3/*
%{_docdir}/%{name}/*
%files plugins
%dir %{_libdir}/%{name}/plugins
%{_libdir}/%{name}/plugins/*.so
%changelog
* Sun Oct 27 2013 Reindl Harald <h....@thelounge.net>
- build with hwloc-support
* Tue Oct 15 2013 Reindl Harald <h....@thelounge.net>
- update to 4.0.2 stable
* Mon Sep 2 2013 Reindl Harald <h....@thelounge.net>
- update to 4.0.1 stable
* Sat Aug 24 2013 Reindl Harald <h....@thelounge.net>
- update to 4.0.0-dev
* Fri Aug 9 2013 Reindl Harald <h....@thelounge.net>
- update to 3.3.5-dev
- move default-configurations to "trafficserver-manpages" package
- move plugins to "trafficserver-plugins" package
- cleanup permissions
- ReadWriteDirectories=/etc/trafficserver/internal
- ReadWriteDirectories=/etc/trafficserver/snapshots
- remove unused files from bindir
- enable-reclaimable-freelist
- enable-linux-native-aio
* Wed Jul 31 2013 Reindl Harald <h....@thelounge.net>
- update to 3.2.5
- add "--with-jemalloc" to configure-flags
* Tue Jun 25 2013 Reindl Harald <h....@thelounge.net>
- fix rpath and add "%{_sysconfdir}/ld.so.conf.d/trafficserver-x86_64.conf"
* Tue Jun 11 2013 Reindl Harald <h....@thelounge.net>
- Cleanup SPEC and hardening build
- split manpages in sub-package
- patch 404-response from "Not Found on Accelerator" to "Not Found" to avoid fingerprinting
Re: ATS - Performance Related Questions - High
Posted by Jean Baptiste Favre <we...@jbfavre.org>.
Hello,
On 09/11/2013 09:13, Omid Kosari wrote:
> Bryan Call wrote
>> Don't compile your own version of ATS and use the packages on dist.
>
> Unfortunately dists are a bit lazy to implement latest versions . How we can
> use 4.x on ubuntu ?
You can backport it :)
But, since Ubuntu is based on Debian and Debian does not provides 4.x
yet, you'll have to adapt packaging from an earlier version. Using 3.3.2
version available in Debian experimental is a good start.
Another solution is to build ATS from source. But then, you'll loose
packaging advantages.
Regards,
Jean-Baptiste
Re: ATS - Performance Related Questions - High
Posted by Omid Kosari <om...@yahoo.com>.
Bryan Call wrote
> Don't compile your own version of ATS and use the packages on dist.
Unfortunately dists are a bit lazy to implement latest versions . How we can
use 4.x on ubuntu ?
--
View this message in context: http://apache-traffic-server.24303.n7.nabble.com/ATS-Performance-Related-Questions-High-tp24p34.html
Sent from the Apache Traffic Server mailing list archive at Nabble.com.
Re: ATS - Performance Related Questions - High
Posted by Bryan Call <bc...@yahoo-inc.com>.
The performance is *much* better on ATS. It really depends on your traffic patterns to determine how much better it will be. Also, the CPU usage is much lower on ATS.
Some groups use RAID10 for ease of operation, but performance will be less then using the disk devices directly.
You can email internal mailing lists and I will give you more details. Don't compile your own version of ATS and use the packages on dist.
-Bryan
On Nov 8, 2013, at 1:31 AM, Karthick S <ka...@gmail.com> wrote:
> Hi Team,
>
> I would like to know few performance related information. We have capacity
> planning meeting for setting up a new cluster with AT. So your input will
> be a great value add in our capacity planning.
>
> Questions:
> 1) YTS(Yahoo Traffic Server) has some performance optimization which
> utilizes the RAID-10 level for effective cache lookup. Does this ATS also
> holds the similar type of feature which will gel well with RAID 10 level.
> If not we will go ahead with RAID 0 to save more disk space for larger
> cache. Currently we have 300GB cache on each ATS nodes.
> 2) How does ATS scales with respect to CPU , Memory, Disk? We have our
> current production running with YTS and 16GB ram. Will ATS scale its
> performance If I am getting 24GB ram ? How ATS will scale with respect to
> hardware. Can you please give some insight about it.
>
> 3) What is the QPS of current ATS?
>
> PS: Please treat this email as high priority, all your inputs will be
> valuable to me.
>
> Regards,
> Karthick
Re: ATS - Performance Related Questions - High
Posted by Bryan Call <bc...@yahoo-inc.com>.
The performance is *much* better on ATS. It really depends on your traffic patterns to determine how much better it will be. Also, the CPU usage is much lower on ATS.
Some groups use RAID10 for ease of operation, but performance will be less then using the disk devices directly.
You can email internal mailing lists and I will give you more details. Don't compile your own version of ATS and use the packages on dist.
-Bryan
On Nov 8, 2013, at 1:31 AM, Karthick S <ka...@gmail.com> wrote:
> Hi Team,
>
> I would like to know few performance related information. We have capacity
> planning meeting for setting up a new cluster with AT. So your input will
> be a great value add in our capacity planning.
>
> Questions:
> 1) YTS(Yahoo Traffic Server) has some performance optimization which
> utilizes the RAID-10 level for effective cache lookup. Does this ATS also
> holds the similar type of feature which will gel well with RAID 10 level.
> If not we will go ahead with RAID 0 to save more disk space for larger
> cache. Currently we have 300GB cache on each ATS nodes.
> 2) How does ATS scales with respect to CPU , Memory, Disk? We have our
> current production running with YTS and 16GB ram. Will ATS scale its
> performance If I am getting 24GB ram ? How ATS will scale with respect to
> hardware. Can you please give some insight about it.
>
> 3) What is the QPS of current ATS?
>
> PS: Please treat this email as high priority, all your inputs will be
> valuable to me.
>
> Regards,
> Karthick