You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Jeremy Utley <je...@gmail.com> on 2012/06/14 21:37:21 UTC
Problems with Squid format logging
Good afternoon everyone!
We're having some issues with Traffic server's squid format log files,
and I'm wondering if anyone else has ever encountered this type of
situation yet. First off, a little background on how we have things
set up:
We're running trafficserver 3.0.4 on CentOS 6.2 installed from the
RPMs in the Redhat EPEL repository. TS is running on the firewall for
our office, acting as a transparent proxy. IPTables is intercepting
all outbound http traffic and redirecting it at trafficserver
listening on port 8080:
[0:0] -A PREROUTING -m state --state NEW,ESTABLISHED,RELATED -m tcp -s
192.168.x.y/255.255.x.y -p tcp --dport 80 -j REDIRECT --to-port 8080
This was to replace an existing squid setup that wasn't performing
quite as well as we'd like, and functionally, it's been working great
for over a week now.
However, when we were using squid, we also had the program "sarg" (1)
doing a daily analysis of our squid logs. So I thought, no problem,
TS has the capability to write logfiles in squid format, we can just
use sarg against those logs and continue on as normal. However, that
is not working. SARG keeps bailing on what looks to be invalid lines
in the log files generated by traffic server. A sample of one of
those lines as displayed by "less squid.log" is shown below:
1339592276.829 40 192.168.x.y TCP_MISS/200 1032 GET http:///_tp/js/JSONRequest.
js - DIRECT/www.bravotv.com application/x-javascript -
CY<CA><FD><83><AE>%18%20%5D%16<E4><B6>Y<C5><D7>t<9F>%1Cp%5D<A2>%07<BA>%0CB<F3>
<E5><82><E7>Iw/l<8F><AA>'%7D<93>-<D6><E7>%0DZ%11%18<BE><85>z<FD>l<B1>&<83>I%10
<C9><F2>%16%02<92>%5E%13<F1><E8>?<C4><E2><A7><F6>PÖ¡09<96>%22<FE><8F><FA><B0><C2>Riw<9B>ß?<98><B5>Y0N<9D>2?<F7><AF>1Ó¤<BB>;<FA><BA><9C>V<8A><C6>FF<8C><A0><D1><C1>B<CE>%01%07s<D9>%1C%13O<C7>.<E0><C9><FE><C6><FE><AA>5<B5><EC>ï<B8>*X<A9><8D>p<D3>%1AF8<82><CC>&%0F<A8>b<A5><92>wV<U+0617>s<FE>S<B1>9<8A>'<8C>U<91>2<F8>v<FC><FC>
<EF>%23<B0><E3>E<A1>x<D9>%10<8B>%1F<CA>n<U+0C5F>%1F<CB>o<F3><AC>Z(<98><A7><EB>
<90><E0><81><F4><B7>%5Ek<E6><94>%1AÒ<A0>b<CC><F7>%14<B5>!k<F4>%09<DD>%22%22Ø¥@
<8A>Bà Ä<F6>/<97>%22æ<E4>ß®%5D%17<F2><91>ì<E1><BA>rßp5<99>%10?qÈ<8E>%5CG<81><B5>.f̹<F3>:%7E<F8>Ce<E7>h(<98>C<BC><B2>9N%07<B0><FE><F3><B1>h<B3><9B>r<90>%04cp<B0>n
<89><E6>c%1Bb<F6><A0>%11<FE><EA><F1><ED>b<C9>L<8F><BC><8C>B<E9>l%5EI7<8E><AD>
<84>7<AE><8D><F6><8C><E5><D3><E5>:<EA>Q%7F%0D<E6>Ò63<A9>%09<FB><B3><C3>×´<9E><9E>'Þ§<F7>Õ<E8>e<F0>Q/<A9><C8>J<ED>,<99>%18ï§<AB><D4><E3>j<F8>m<94>5<E2>X&<F4><DD>_
<AC>Ç=FyX<F8><AF><E5>%5Dq2<ED>K<9D>%0C;<B4>8%25<B2><CC><DE>P<81>%01%7E<C6>%60.
<AF>Z%1Ak<9F><86><F6>Ö«<CD>A%10<FC><D6>L<92>%17%03Y+h<EF>%10v<C1><F8>h<F1>T%7C%07
<D9><F1><A6>:2%08<9C><FC>%06<AA>%20<B2><88><C7><CE>%20%5D%7E'<A4>+<80><80><9C>J3<FB>%3EA%7B<FA>Lj<8D><F8>%5C<CD>T<87><DC>0<B7>IQ*<CF>IA<9C>9x<B2><99>V<8C><93>
<AE><9D><A5><98>%09<EE>B<9A>p<B9><CB><D9><EC>$<FF><D1>3<EC>%22%10o<9F>)<80><EB>
<FF>)0<D1><DF><C6><F1><80>i<90><85><E5><90>'l<B9><96>P<84>%02<EA><FB><F0><A4>
<CE><E9><EE><FB><EF>l<C9>=<A4>L<AE>%22L%0C<D7>%0Fr%0B%20F)<AA>%5C
INVALID_CODE(45)/1 - text/html
Also of note, the "hex" characters within <> is hilited when looking
at it in less.
Has anyone ever seen output like this from the squid format logs
generated by traffic server? Any way to solve this problem?
Thanks for any help anyone can give!
--
Jeremy Utley
Re: Problems with Squid format logging
Posted by Nick Berry <nb...@linkedin.com>.
Is this squid.log or squid.blog? The latter is a binary log and needs to be decoded by traffic_logcat.
example...
traffic_logcat /path/to/squid.blog
or
cat /path/to/squid.blog | traffic_logcat
On Jun 14, 2012, at 12:37 PM, Jeremy Utley <je...@gmail.com> wrote:
> Good afternoon everyone!
>
> We're having some issues with Traffic server's squid format log files,
> and I'm wondering if anyone else has ever encountered this type of
> situation yet. First off, a little background on how we have things
> set up:
>
> We're running trafficserver 3.0.4 on CentOS 6.2 installed from the
> RPMs in the Redhat EPEL repository. TS is running on the firewall for
> our office, acting as a transparent proxy. IPTables is intercepting
> all outbound http traffic and redirecting it at trafficserver
> listening on port 8080:
>
> [0:0] -A PREROUTING -m state --state NEW,ESTABLISHED,RELATED -m tcp -s
> 192.168.x.y/255.255.x.y -p tcp --dport 80 -j REDIRECT --to-port 8080
>
> This was to replace an existing squid setup that wasn't performing
> quite as well as we'd like, and functionally, it's been working great
> for over a week now.
>
> However, when we were using squid, we also had the program "sarg" (1)
> doing a daily analysis of our squid logs. So I thought, no problem,
> TS has the capability to write logfiles in squid format, we can just
> use sarg against those logs and continue on as normal. However, that
> is not working. SARG keeps bailing on what looks to be invalid lines
> in the log files generated by traffic server. A sample of one of
> those lines as displayed by "less squid.log" is shown below:
>
>
>
> 1339592276.829 40 192.168.x.y TCP_MISS/200 1032 GET http:///_tp/js/JSONRequest.
> js - DIRECT/www.bravotv.com application/x-javascript -
> CY<CA><FD><83><AE>%18%20%5D%16<E4><B6>Y<C5><D7>t<9F>%1Cp%5D<A2>%07<BA>%0CB<F3>
> <E5><82><E7>Iw/l<8F><AA>'%7D<93>-<D6><E7>%0DZ%11%18<BE><85>z<FD>l<B1>&<83>I%10
> <C9><F2>%16%02<92>%5E%13<F1><E8>?<C4><E2><A7><F6>PÖ¡09<96>%22<FE><8F><FA><B0><C2>Riw<9B>ß?<98><B5>Y0N<9D>2?<F7><AF>1Ó¤<BB>;<FA><BA><9C>V<8A><C6>FF<8C><A0><D1><C1>B<CE>%01%07s<D9>%1C%13O<C7>.<E0><C9><FE><C6><FE><AA>5<B5><EC>ï<B8>*X<A9><8D>p<D3>%1AF8<82><CC>&%0F<A8>b<A5><92>wV<U+0617>s<FE>S<B1>9<8A>'<8C>U<91>2<F8>v<FC><FC>
> <EF>%23<B0><E3>E<A1>x<D9>%10<8B>%1F<CA>n<U+0C5F>%1F<CB>o<F3><AC>Z(<98><A7><EB>
> <90><E0><81><F4><B7>%5Ek<E6><94>%1AÒ<A0>b<CC><F7>%14<B5>!k<F4>%09<DD>%22%22Ø¥@
> <8A>Bà Ä<F6>/<97>%22æ<E4>ß®%5D%17<F2><91>ì<E1><BA>rßp5<99>%10?qÈ<8E>%5CG<81><B5>.f̹<F3>:%7E<F8>Ce<E7>h(<98>C<BC><B2>9N%07<B0><FE><F3><B1>h<B3><9B>r<90>%04cp<B0>n
> <89><E6>c%1Bb<F6><A0>%11<FE><EA><F1><ED>b<C9>L<8F><BC><8C>B<E9>l%5EI7<8E><AD>
> <84>7<AE><8D><F6><8C><E5><D3><E5>:<EA>Q%7F%0D<E6>Ò63<A9>%09<FB><B3><C3>×´<9E><9E>'Þ§<F7>Õ<E8>e<F0>Q/<A9><C8>J<ED>,<99>%18ï§<AB><D4><E3>j<F8>m<94>5<E2>X&<F4><DD>_
> <AC>Ç=FyX<F8><AF><E5>%5Dq2<ED>K<9D>%0C;<B4>8%25<B2><CC><DE>P<81>%01%7E<C6>%60.
> <AF>Z%1Ak<9F><86><F6>Ö«<CD>A%10<FC><D6>L<92>%17%03Y+h<EF>%10v<C1><F8>h<F1>T%7C%07
> <D9><F1><A6>:2%08<9C><FC>%06<AA>%20<B2><88><C7><CE>%20%5D%7E'<A4>+<80><80><9C>J3<FB>%3EA%7B<FA>Lj<8D><F8>%5C<CD>T<87><DC>0<B7>IQ*<CF>IA<9C>9x<B2><99>V<8C><93>
> <AE><9D><A5><98>%09<EE>B<9A>p<B9><CB><D9><EC>$<FF><D1>3<EC>%22%10o<9F>)<80><EB>
> <FF>)0<D1><DF><C6><F1><80>i<90><85><E5><90>'l<B9><96>P<84>%02<EA><FB><F0><A4>
> <CE><E9><EE><FB><EF>l<C9>=<A4>L<AE>%22L%0C<D7>%0Fr%0B%20F)<AA>%5C
> INVALID_CODE(45)/1 - text/html
>
> Also of note, the "hex" characters within <> is hilited when looking
> at it in less.
>
> Has anyone ever seen output like this from the squid format logs
> generated by traffic server? Any way to solve this problem?
>
> Thanks for any help anyone can give!
>
> --
> Jeremy Utley