You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by JungHwan Lim <li...@gmail.com> on 2011/11/02 08:20:09 UTC
TS cache Dynamic Contents with cacheurl plugin
Dear TS,
I want TS to cache dynamic content as Youtube.
So, I confiugred recode.config file as follow.
"CONFIG proxy.config.http.cache.cache_urls_that_look_dynamic INT 1 "
then TS cache and serve youtube video to user1, but TS don't serve user2.
I found one mail from mail archive that is " how to cache object with
different urls (with question mark) just once?"
so, I install cacheurl plugin.
but cacheurl plugin seems not working because I see origin youtube url in
squid.log
I need your advise.
I instruments ATS 3.0.1 (forward proxy mode) in Ubuntu 11
I installed cacheurl plugin.
*[root@localhost cacheurl]# pwd
/usr/local/bin/cacheurl
[root@localhost cacheurl]# ls
cacheurl.c cacheurl.config.example cacheurl.lo cacheurl.so CHAGES
CHANGES LICENSE Makefile NOTICE README
[root@localhost cacheurl]#*
I add regex for youtube to cacheurl.config
*[root@localhost local]# vi libexec/trafficserver/cacheurl.config*
*# Configuration file format:
# Configuration file format:
#
# url_pattern cache_key_replacement
#
# The url_pattern is a regular expression (pcre). The replacement can
contain
# $1, $2 and so on, which will be replaced with the appropriate matching
group
# from the pattern.
# Make files from s1.example.com, s2.example.com and s3.example.com all
# be cached with the same key.
# Adding a unique suffix (TSINTERNAL in this example) to the cache key
# guarantees that it won't clash with a real URL should s.example.com exist.
#http://s[123].example.com/(.*)
http://s.example.com.TSINTERNAL/$1<http://s.example.com.tsinternal/$1>
# Cache based on only some parts of a query string (e.g. ignore session
# information). This plucks out the id and format query string variables and
# only considers those when making the cache key.
#http://www.example.com/video\?.*?\&?(id=[0-9a-f]*).*?\&(format=[a-z]*)
http://video-srv.example.com.ATSINTERNAL/$1&$2<http://video-srv.example.com.atsinternal/$1&$2>
# Completely ignore a query string for a specific page
#http://www.example.com/some/page.html(?:\?|$)
http://www.example.com/some/page.html
#http:\/\/(.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com)\/(get_video|videoplayback|videodownload)\?.*?\&(itag=[0-9]*).*?\&(id=[a-zA-Z0-9]*)
http://video-srv
.youtube.com.ATSINTERNAL/$3&$4
http://o-o.preferred.hkg05s01.(v[0-9]*).lscache[0-9]).
c.youtube.com/videoplayback\?.* <http://c.youtube.com/videoplayback%5C?.*>
?\&(itag=[0-9]*).*?\&(id=[a-zA-Z0-9]*)
http://video-srv.youtube.com.ATSINTERNAL/$3&$4<http://video-srv.youtube.com.atsinternal/$3&$4>
*
and add cacheurl.so to plugin.config
I added 3 line but what of all is correct?
*[root@localhost local]# vi etc/trafficserver/plugin.config*
*# Comments start with a '#' and continue to the end of the line
# Blank lines are ignored
#
# On UNIX
# test-plugin.so arg1 arg2 arg3
#
# On WIN NT/2000
# test-plugin.dll arg1 arg2 arg3
#inktomi/iwx/iwx.so
#inktomi/abuse/abuse.so etc/trafficserver/abuse.config
#inktomi/icx/icx.so etc/trafficserver/icx.config
cacheurl.so
cacheurl.so /usr/local/libexec/trafficserver/cacheurl.config
/usr/local/libexec/trafficserver/cacheurl.so
/usr/local/libexec/trafficserver/cacheurl.config*
last,
In squid.log, I found youtube url not changed.
*[root@localhost local]# vi var/log/trafficserver/squid.log
1320207814.109 915 168.219.208.213 TCP_HIT/200 1726867 GET
http://o-o.preferred.hkg05s01.v12.lscache8.c.youtube.com/videoplayback?sparams=id%252Cexpire%252Cip%252Cipbits%25
2Citag%252Csource%252Calgorithm%252Cburst%252Cfactor%252Ccp&fexp=904442%252C912600&algorithm=throttle-factor&itag=34&ip=165.0.0.0&burst=40&sver=3&signature=B48AE121D4E46657
6C8DD7A0E74383096C79D3E1.7390E7F468A8628467DD5906E5A01C2592DC1C6C&source=youtube&expire=1320231600&key=yt1&ipbits=8&factor=1.25&cp=U0hRRVJOT19FSkNOMV9KSFNHOldhUDcxWlQ4Smhr&
id=e5b6b4b59fcfe5c8 - NONE/- video/x-flv -*
*
*
Thanks in advance,
Re: TS cache Dynamic Contents with cacheurl plugin
Posted by Mark Harrison <ma...@omniti.com>.
The cacheurl plugin only changes what URL is used to identify whether
the object is in cache or not, it doesn't change the URL requests, and
so doesn't change the URL showed in the squid logs. There should be a
cacheurl.log file that logs all cacheurl rewrites (the messages show
'Rewriting cache URL from http://foo/ to http://bar/'), and should
show whether your rules are working or not.
As for how to test if your rules are actually working or not, you
should have two computers. Visit a youtube video with the first
computer and wait for the video to finish loading, then view the same
video on the second computer - it should use a slightly different URL
to request the video, but the video should finish loading almost
immediately (you can see this on the youtube video time slider). In
addition, you should see that the relevant entry in the squid logs
show a cache hit.
If none of that works, check the logs (or stdout) to see if there were
any errors with the plugin.
On Wed, Nov 2, 2011 at 3:20 AM, JungHwan Lim <li...@gmail.com> wrote:
> Dear TS,
>
> I want TS to cache dynamic content as Youtube.
> So, I confiugred recode.config file as follow.
> "CONFIG proxy.config.http.cache.cache_urls_that_look_dynamic INT 1 "
> then TS cache and serve youtube video to user1, but TS don't serve user2.
>
> I found one mail from mail archive that is " how to cache object with
> different urls (with question mark) just once?"
> so, I install cacheurl plugin.
> but cacheurl plugin seems not working because I see origin youtube url in
> squid.log
>
> I need your advise.
>
> I instruments ATS 3.0.1 (forward proxy mode) in Ubuntu 11
>
> I installed cacheurl plugin.
> [root@localhost cacheurl]# pwd
> /usr/local/bin/cacheurl
> [root@localhost cacheurl]# ls
> cacheurl.c cacheurl.config.example cacheurl.lo cacheurl.so CHAGES
> CHANGES LICENSE Makefile NOTICE README
> [root@localhost cacheurl]#
>
> I add regex for youtube to cacheurl.config
> [root@localhost local]# vi libexec/trafficserver/cacheurl.config
> # Configuration file format:
> # Configuration file format:
> #
> # url_pattern cache_key_replacement
> #
> # The url_pattern is a regular expression (pcre). The replacement can
> contain
> # $1, $2 and so on, which will be replaced with the appropriate matching
> group
> # from the pattern.
>
> # Make files from s1.example.com, s2.example.com and s3.example.com all
> # be cached with the same key.
> # Adding a unique suffix (TSINTERNAL in this example) to the cache key
> # guarantees that it won't clash with a real URL should s.example.com exist.
> #http://s[123].example.com/(.*) http://s.example.com.TSINTERNAL/$1
>
> # Cache based on only some parts of a query string (e.g. ignore session
> # information). This plucks out the id and format query string variables and
> # only considers those when making the cache key.
> #http://www.example.com/video\?.*?\&?(id=[0-9a-f]*).*?\&(format=[a-z]*) http://video-srv.example.com.ATSINTERNAL/$1&$2
>
> # Completely ignore a query string for a specific page
> #http://www.example.com/some/page.html(?:\?|$) http://www.example.com/some/page.html
>
> #http:\/\/(.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com)\/(get_video|videoplayback|videodownload)\?.*?\&(itag=[0-9]*).*?\&(id=[a-zA-Z0-9]*) http://video-srv
> .youtube.com.ATSINTERNAL/$3&$4
>
> http://o-o.preferred.hkg05s01.(v[0-9]*).lscache[0-9]).c.youtube.com/videoplayback\?.*?\&(itag=[0-9]*).*?\&(id=[a-zA-Z0-9]*) http://video-srv.youtube.com.ATSINTERNAL/$3&$4
>
>
> and add cacheurl.so to plugin.config
> I added 3 line but what of all is correct?
> [root@localhost local]# vi etc/trafficserver/plugin.config
> # Comments start with a '#' and continue to the end of the line
> # Blank lines are ignored
> #
> # On UNIX
> # test-plugin.so arg1 arg2 arg3
> #
> # On WIN NT/2000
> # test-plugin.dll arg1 arg2 arg3
> #inktomi/iwx/iwx.so
> #inktomi/abuse/abuse.so etc/trafficserver/abuse.config
> #inktomi/icx/icx.so etc/trafficserver/icx.config
> cacheurl.so
> cacheurl.so /usr/local/libexec/trafficserver/cacheurl.config
> /usr/local/libexec/trafficserver/cacheurl.so
> /usr/local/libexec/trafficserver/cacheurl.config
>
>
>
> last,
> In squid.log, I found youtube url not changed.
> [root@localhost local]# vi var/log/trafficserver/squid.log
> 1320207814.109 915 168.219.208.213 TCP_HIT/200 1726867
> GET http://o-o.preferred.hkg05s01.v12.lscache8.c.youtube.com/videoplayback?sparams=id%252Cexpire%252Cip%252Cipbits%25
> 2Citag%252Csource%252Calgorithm%252Cburst%252Cfactor%252Ccp&fexp=904442%252C912600&algorithm=throttle-factor&itag=34&ip=165.0.0.0&burst=40&sver=3&signature=B48AE121D4E46657
> 6C8DD7A0E74383096C79D3E1.7390E7F468A8628467DD5906E5A01C2592DC1C6C&source=youtube&expire=1320231600&key=yt1&ipbits=8&factor=1.25&cp=U0hRRVJOT19FSkNOMV9KSFNHOldhUDcxWlQ4Smhr&
> id=e5b6b4b59fcfe5c8 - NONE/- video/x-flv -
>
>
> Thanks in advance,