You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by "Jay Li (jianli)" <ji...@cisco.com> on 2014/10/20 09:31:55 UTC
the questions on the usage of cacheurl plugin
Hi,
I want to cache some url based on its keyword of the url. So I refer to the cacheurl from https://docs.trafficserver.apache.org/en/latest/reference/plugins/cacheurl.en.html. But I had the following questions after I read the description.
1. Is the cacheurl built by default ? The website said "This plugin is only built if the configure option --enable-experimental-plugins”. But I had the cacheurl.so available under my libexec/trafficserver (/home/ats/test/local/trafficserver-5.0.1/libexec/trafficserver). So I think it was built by default in 5.0.1, right?
2. Where is the cacheurl.config file located?
The website said “ Create a cacheurl.config file in the plugin directory with the url regex patterns to match”. But there is no plugin folder under my installed directory. Should I put the cacheurl.config under the folder same as cacheurl.so lacerated ? (i.e. /home/ats/test/local/trafficserver-5.0.1/libexec/trafficserver)
3. What is the correct reg url for my links ( itag and id are keywords)?
My url for example
1. http://10.201.12.17/videoplayback?Body-Bytes=15000001&clen=101715833&itag=139&range=0-15000000&id=ZZZZccccddddeee15952&aaa?Transaction-Profile=YT_100MB
2. http://10.201.12.32/videoplayback?Body-Bytes=5000000&clen=101717619&itag=138&range=15000001-20000000&id=ZZZZccccddddeee16232&aaa?Transaction-Profile=YT_100MB
Is the following regurl correct?
http://[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}/videoplayback?Body-Bytes=\?.*\&(itag=[0-9]*).*\&(id=[0-9a-zA-Z]*).* http://www.myvideo.com/$1&$2
Thank you!
Regards
Jay
http://[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}[^&]*/f4v/.*id=tudou.itemid\=([0-9]*).* http://www.tudou.com/$1
http://[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}[^&]*/flv/.*id=tudou.itemid\=([0-9]*).* http://www.tudou.com/$1
http://[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}/youku/.*/(.*-.*-.*-.*-[^?]*)(.*) http://www.youku.com/$1
http://[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}/sohu/[0-9]*/[0-9]*/[0-9]*/(.*).mp4?key=.* http://tv.sohu.com/$1.mp4
http://.*\..*\..*\..*/.*\.com/flvdownload/[[:digit:]]{1,3}/[[:digit:]]{1,3}/([^?]*)(.*) http://www.56.com/$1
http://[[:digit:]]{1,3}/mp4files/.*/.*\.com/images/tuiguang/[[:digit:]]{6,6}/(.*\.mp4) http://www.56.com/$1
http://.*\..*\.com/images/tuiguang/([[:digit:]]{6,6})/(.*\.mp4) http://www.56.com/tuiguang/$1/$2
http://[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}/mov.bn.netease.com/.*/.*/.*/.*/.*/([^?]*)(.*) http://v.163.com/$1
http://[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}/.*-.*-.*/.*/cemov.bn.netease.com/.*/.*/.*/.*/.*/([^?]*)(.*) http://v.163.com/$1
Regards
Jay
Re: the questions on the usage of cacheurl plugin
Posted by James Peach <jp...@apache.org>.
> On Oct 27, 2014, at 11:03 AM, Alan M. Carroll <am...@network-geographics.com> wrote:
>
> Monday, October 20, 2014, 12:31:55 AM, you wrote:
>
>> 1. Is the cacheurl built by default ? The website said "This plugin is only built if the configure option --enable-experimental-plugins”. But I had the cacheurl.so available under my libexec/trafficserver (/home/ats/test/local/trafficserver-5.0.1/libexec/trafficserver). So I think it was built by default in 5.0.1, right?
>
> Yes, that seems to be the case. The plugins in the "plugins" directory should be built by default. The plugins builds enabled by --enable-experimental-plugins are in "plugins/experimental". cacheurl was there first (as usual) and was at some point promoted to a standard default plugin but the documentation wasn't updated.
>
>> 2. Where is the cacheurl.config file located? The website said “ Create a cacheurl.config file in the plugin directory with the url regex patterns to match”. But there is no plugin folder under my installed directory. Should I put the cacheurl.config under the folder same as cacheurl.so lacerated ? (i.e. /home/ats/test/local/trafficserver-5.0.1/libexec/trafficserver)
>
> Yes, you can put it there. I find it a bit of an odd place to look, but that does appear to be what the code does.
IIRC that was done for backwards compatibility.
> You can also pass the config file path as the first argument in the plugins.config file for the plugin, e.g.
>
> cacheurl.so jay-cacheurl.config
>
> with the argument being interpreted as relative to the config file directory (where records.config is).
Yep, this is preferable imho
J
Re: the questions on the usage of cacheurl plugin
Posted by "Alan M. Carroll" <am...@network-geographics.com>.
Monday, October 20, 2014, 12:31:55 AM, you wrote:
> 1. Is the cacheurl built by default ? The website said "This plugin is only built if the configure option --enable-experimental-plugins”. But I had the cacheurl.so available under my libexec/trafficserver (/home/ats/test/local/trafficserver-5.0.1/libexec/trafficserver). So I think it was built by default in 5.0.1, right?
Yes, that seems to be the case. The plugins in the "plugins" directory should be built by default. The plugins builds enabled by --enable-experimental-plugins are in "plugins/experimental". cacheurl was there first (as usual) and was at some point promoted to a standard default plugin but the documentation wasn't updated.
> 2. Where is the cacheurl.config file located? The website said “ Create a cacheurl.config file in the plugin directory with the url regex patterns to match”. But there is no plugin folder under my installed directory. Should I put the cacheurl.config under the folder same as cacheurl.so lacerated ? (i.e. /home/ats/test/local/trafficserver-5.0.1/libexec/trafficserver)
Yes, you can put it there. I find it a bit of an odd place to look, but that does appear to be what the code does.
You can also pass the config file path as the first argument in the plugins.config file for the plugin, e.g.
cacheurl.so jay-cacheurl.config
with the argument being interpreted as relative to the config file directory (where records.config is). Or you can use an absolute path.
> 3. What is the correct reg url for my links ( itag and id are keywords)?
> My url for example
>
> 1. http://10.201.12.17/videoplayback?Body-Bytes=15000001&clen=101715833&itag=139&range=0-15000000&id=ZZZZccccddddeee15952&aaa?Transaction-Profile=YT_100MB
> 2. http://10.201.12.32/videoplayback?Body-Bytes=5000000&clen=101717619&itag=138&range=15000001-20000000&id=ZZZZccccddddeee16232&aaa?Transaction-Profile=YT_100MB
>
> Is the following regurl correct?
> http://[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}/videoplayback?Body-Bytes=\?.*\&(itag=[0-9]*).*\&(id=[0-9a-zA-Z]*).* http://www.myvideo.com/$1&$2
>
I haven't actually used this plugin, so I'm not sure but it seems reasonable. You might try "http://(?:[[:digit:]]{1,3}){4}" as the leading part.
A couple of side notes -
1) The result must be syntactically a URL but it is, in the end, just a string as far as the cache is concerned. So you could use a result like "video://jay/$1&$2". This is what I have done in other efforts like this to be sure to avoid any collisions with a real URL.
2) These requests appear to be range based requests. If you don't store the range part as well, you may get some rather unexpected results when you read the URL back from cache. That is, something with a query parameter of "range=0-1500000" would match a URL the same tag and id and a range parameter of "1500001-2000000".