You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by Randeep <ra...@gmail.com> on 2016/10/27 07:22:49 UTC
Query strings are not forwarded
Hi all,
We are using ats to cache our live MPEG dash and HLS streams.
We use version 6.2.0 with parent-child architecture.
In the requests there are some query strings like timestamps for playing
catchup,start-over,timeshift etc.
Before the requests where not caching so we added @pparam=no-query-string
so that it will cache without query stings. I thought it will strip the
query strings only when caching but it is stripping the query-string while
forwarding to domain also. Because of this a lot of services like catchup
doesn't work.
remap.config
map http://jitp1.domain.com/ http://bglrcache1.domain.com/
@plugin=/opt/ats/libexec/trafficserver/regex_remap.so
@pparam=/opt/ats/etc/trafficserver/jitp1.config @pparam=no-query-string
cat /opt/ats/etc/trafficserver/jitp5.config
^/(.*)$ http://bglrcache5.domain.com/$1
How can we achieve both url forwarding with query sting and caching the
content without the query-string?
--
Randeep
Mob: +919447831699[kerala]
Mob: +919880050349[B'lore]
http://twitter.com/Randeeppr
http://in.linkedin.com/in/randeeppr
[image: --]
Randeep Raman
[image: http://]about.me/Randeeppr
<http://about.me/Randeeppr>
Re: Query strings are not forwarded
Posted by Randeep <ra...@gmail.com>.
Hi,
I used the cachekey plugin as below. But its not working. Am I doing it
wrong?
My remap.config
map http://jitp1.domain.com/ http://bglrcache1.domain.com/
@pparam=--ua-whitelist=lukupplayer:/opt/ats/etc/trafficserver/randeep.config
@pparam=--ua-capture=(stagefright\/[^\s]*).*
cat /opt/ats/etc/trafficserver/randeep.config
^stagefright/1.*
I am requesting the like this
curl "http://jitp1.domain.com/dashw/abc/manifest.mpd" -H 'User-Agent:
stagefright/1.2 (Linux;Android 4.2.109)' -H 'X-Debug: X-Cache-Key'
But its not rejected!
[root@cachetest ~]# tail -f /opt/ats/var/log/trafficserver/squid.log |grep
abc
1477636406.439 3 172.16.0.51 TCP_MISS/404 199 GET
http://bglrcache1.domain.com/dashw/abc/manifest.mpd - PARENT_HIT/172.16.0.40
text/plain
Please guide.
Regards,
Randeep
On Thu, Oct 27, 2016 at 5:02 PM, Randeep <ra...@gmail.com> wrote:
> Hi,
>
> url http://jitp1.dmain.com/dashw/abpnews/manifest.mpd
> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime=1477454400000&stoptime=1477456200000>
> will give the live content
>
> url http://jitp1.dmain.com/dashw/abpnews/manifest.mpd?
> starttime=1477454400000&stoptime=1477456200000
> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime=1477454400000&stoptime=1477456200000>
> will give the recorded content from the starttime to endtime. The manifest
> file will be for the specified time.
>
> but in that case if we cache the response without query strings for the
> above, it might conflict with the live requests!
>
> I think its better not cache the manifest url with query strings. Only the
> video chunks and manifest file without query string we will cache.
> Is that possible?
>
> Regards,
> Randeep
>
> On Thu, Oct 27, 2016 at 4:50 PM, Reindl Harald <h....@thelounge.net>
> wrote:
>
>>
>>
>> Am 27.10.2016 um 13:05 schrieb Randeep:
>>
>>> Our urls are like
>>>
>>> http://jitp1.dmain.com/dashw/abpnews/manifest.mpd?starttime=
>>> 1477454400000&stoptime=1477456200000
>>> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime
>>> =1477454400000&stoptime=1477456200000>
>>>
>>> We need to cache only
>>> http://jitp1.dmain.com/dashw/abpnews/manifest.mpd
>>> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime
>>> =1477454400000&stoptime=1477456200000>
>>>
>>
>> but *why* are your URLs than that way?
>>
>> if that params don't do anything remove them
>>
>> if you configrue your proxy to ignore params and always deliver the same
>> content that will for sure break something else and in doubt it will break
>> terrible
>>
>
>
>
> --
> Randeep
> Mob: +919447831699[kerala]
> Mob: +919880050349[B'lore]
> http://twitter.com/Randeeppr
> http://in.linkedin.com/in/randeeppr
>
>
> [image: --]
> Randeep Raman
> [image: http://]about.me/Randeeppr
> <http://about.me/Randeeppr>
>
>
--
Randeep
Mob: +919447831699[kerala]
Mob: +919880050349[B'lore]
http://twitter.com/Randeeppr
http://in.linkedin.com/in/randeeppr
[image: --]
Randeep Raman
[image: http://]about.me/Randeeppr
<http://about.me/Randeeppr>
Re: Query strings are not forwarded
Posted by Randeep <ra...@gmail.com>.
Hi,
I used the cachekey plugin as below. But its not working. Am I doing it
wrong?
My remap.config
map http://jitp1.domain.com/ http://bglrcache1.domain.com/
@pparam=--ua-whitelist=lukupplayer:/opt/ats/etc/trafficserver/randeep.config
@pparam=--ua-capture=(stagefright\/[^\s]*).*
cat /opt/ats/etc/trafficserver/randeep.config
^stagefright/1.*
I am requesting the like this
curl "http://jitp1.domain.com/dashw/abc/manifest.mpd" -H 'User-Agent:
stagefright/1.2 (Linux;Android 4.2.109)' -H 'X-Debug: X-Cache-Key'
But its not rejected!
[root@cachetest ~]# tail -f /opt/ats/var/log/trafficserver/squid.log |grep
abc
1477636406.439 3 172.16.0.51 TCP_MISS/404 199 GET
http://bglrcache1.domain.com/dashw/abc/manifest.mpd - PARENT_HIT/172.16.0.40
text/plain
Please guide.
Regards,
Randeep
On Thu, Oct 27, 2016 at 5:02 PM, Randeep <ra...@gmail.com> wrote:
> Hi,
>
> url http://jitp1.dmain.com/dashw/abpnews/manifest.mpd
> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime=1477454400000&stoptime=1477456200000>
> will give the live content
>
> url http://jitp1.dmain.com/dashw/abpnews/manifest.mpd?
> starttime=1477454400000&stoptime=1477456200000
> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime=1477454400000&stoptime=1477456200000>
> will give the recorded content from the starttime to endtime. The manifest
> file will be for the specified time.
>
> but in that case if we cache the response without query strings for the
> above, it might conflict with the live requests!
>
> I think its better not cache the manifest url with query strings. Only the
> video chunks and manifest file without query string we will cache.
> Is that possible?
>
> Regards,
> Randeep
>
> On Thu, Oct 27, 2016 at 4:50 PM, Reindl Harald <h....@thelounge.net>
> wrote:
>
>>
>>
>> Am 27.10.2016 um 13:05 schrieb Randeep:
>>
>>> Our urls are like
>>>
>>> http://jitp1.dmain.com/dashw/abpnews/manifest.mpd?starttime=
>>> 1477454400000&stoptime=1477456200000
>>> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime
>>> =1477454400000&stoptime=1477456200000>
>>>
>>> We need to cache only
>>> http://jitp1.dmain.com/dashw/abpnews/manifest.mpd
>>> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime
>>> =1477454400000&stoptime=1477456200000>
>>>
>>
>> but *why* are your URLs than that way?
>>
>> if that params don't do anything remove them
>>
>> if you configrue your proxy to ignore params and always deliver the same
>> content that will for sure break something else and in doubt it will break
>> terrible
>>
>
>
>
> --
> Randeep
> Mob: +919447831699[kerala]
> Mob: +919880050349[B'lore]
> http://twitter.com/Randeeppr
> http://in.linkedin.com/in/randeeppr
>
>
> [image: --]
> Randeep Raman
> [image: http://]about.me/Randeeppr
> <http://about.me/Randeeppr>
>
>
--
Randeep
Mob: +919447831699[kerala]
Mob: +919880050349[B'lore]
http://twitter.com/Randeeppr
http://in.linkedin.com/in/randeeppr
[image: --]
Randeep Raman
[image: http://]about.me/Randeeppr
<http://about.me/Randeeppr>
Re: Query strings are not forwarded
Posted by Randeep <ra...@gmail.com>.
Hi,
url http://jitp1.dmain.com/dashw/abpnews/manifest.mpd
<http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime=1477454400000&stoptime=1477456200000>
will
give the live content
url http://jitp1.dmain.com/dashw/abpnews/manifest.mpd?starttime=
1477454400000&stoptime=1477456200000
<http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime=1477454400000&stoptime=1477456200000>
will give the recorded content from the starttime to endtime. The manifest
file will be for the specified time.
but in that case if we cache the response without query strings for the
above, it might conflict with the live requests!
I think its better not cache the manifest url with query strings. Only the
video chunks and manifest file without query string we will cache.
Is that possible?
Regards,
Randeep
On Thu, Oct 27, 2016 at 4:50 PM, Reindl Harald <h....@thelounge.net>
wrote:
>
>
> Am 27.10.2016 um 13:05 schrieb Randeep:
>
>> Our urls are like
>>
>> http://jitp1.dmain.com/dashw/abpnews/manifest.mpd?starttime=
>> 1477454400000&stoptime=1477456200000
>> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime
>> =1477454400000&stoptime=1477456200000>
>>
>> We need to cache only
>> http://jitp1.dmain.com/dashw/abpnews/manifest.mpd
>> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime
>> =1477454400000&stoptime=1477456200000>
>>
>
> but *why* are your URLs than that way?
>
> if that params don't do anything remove them
>
> if you configrue your proxy to ignore params and always deliver the same
> content that will for sure break something else and in doubt it will break
> terrible
>
--
Randeep
Mob: +919447831699[kerala]
Mob: +919880050349[B'lore]
http://twitter.com/Randeeppr
http://in.linkedin.com/in/randeeppr
[image: --]
Randeep Raman
[image: http://]about.me/Randeeppr
<http://about.me/Randeeppr>
Re: Query strings are not forwarded
Posted by Reindl Harald <h....@thelounge.net>.
Am 27.10.2016 um 13:05 schrieb Randeep:
> Our urls are like
>
> http://jitp1.dmain.com/dashw/abpnews/manifest.mpd?starttime=1477454400000&stoptime=1477456200000
> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime=1477454400000&stoptime=1477456200000>
>
> We need to cache only
> http://jitp1.dmain.com/dashw/abpnews/manifest.mpd
> <http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime=1477454400000&stoptime=1477456200000>
but *why* are your URLs than that way?
if that params don't do anything remove them
if you configrue your proxy to ignore params and always deliver the same
content that will for sure break something else and in doubt it will
break terrible
Re: Query strings are not forwarded
Posted by Randeep <ra...@gmail.com>.
Hi,
Our urls are like
http://jitp1.dmain.com/dashw/abpnews/manifest.mpd?starttime=1477454400000&
stoptime=1477456200000
<http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime=1477454400000&stoptime=1477456200000>
We need to cache only
http://jitp1.dmain.com/dashw/abpnews/manifest.mpd
<http://jitp1.lukup.com/dashw/abpnews/manifest.mpd?starttime=1477454400000&stoptime=1477456200000>
I'll try the plugin you recommended.
Regards,
Randeep
On Thu, Oct 27, 2016 at 4:03 PM, Sudheer Vinukonda <
sudheervinukonda@yahoo.com> wrote:
> You may try using cachekey plugin to ignore the query params only from the
> cache key and not from remap, instead of using regex_remap with
> no-query-string param, which will basically remove the query string from
> remap.
>
> There is an excellent documentation and some examples on how to do this at
> https://docs.trafficserver.apache.org/en/latest/admin-
> guide/plugins/cachekey.en.html
>
> - Sudheer
>
> On Oct 27, 2016, at 12:22 AM, Randeep <ra...@gmail.com> wrote:
>
> Hi all,
>
> We are using ats to cache our live MPEG dash and HLS streams.
> We use version 6.2.0 with parent-child architecture.
>
> In the requests there are some query strings like timestamps for playing
> catchup,start-over,timeshift etc.
>
> Before the requests where not caching so we added @pparam=no-query-string
> so that it will cache without query stings. I thought it will strip the
> query strings only when caching but it is stripping the query-string while
> forwarding to domain also. Because of this a lot of services like catchup
> doesn't work.
>
> remap.config
> map http://jitp1.domain.com/ http://bglrcache1.domain.com/
> @plugin=/opt/ats/libexec/trafficserver/regex_remap.so
> @pparam=/opt/ats/etc/trafficserver/jitp1.config @pparam=no-query-string
>
> cat /opt/ats/etc/trafficserver/jitp5.config
> ^/(.*)$ http://bglrcache5.domain.com/$1
>
> How can we achieve both url forwarding with query sting and caching the
> content without the query-string?
>
> --
> Randeep
> Mob: +919447831699[kerala]
> Mob: +919880050349[B'lore]
> http://twitter.com/Randeeppr
> http://in.linkedin.com/in/randeeppr
>
>
> [image: --]
> Randeep Raman
> [image: http://]about.me/Randeeppr
> <http://about.me/Randeeppr>
>
>
--
Randeep
Mob: +919447831699[kerala]
Mob: +919880050349[B'lore]
http://twitter.com/Randeeppr
http://in.linkedin.com/in/randeeppr
[image: --]
Randeep Raman
[image: http://]about.me/Randeeppr
<http://about.me/Randeeppr>
Re: Query strings are not forwarded
Posted by Sudheer Vinukonda <su...@yahoo.com.INVALID>.
You may try using cachekey plugin to ignore the query params only from the cache key and not from remap, instead of using regex_remap with no-query-string param, which will basically remove the query string from remap.
There is an excellent documentation and some examples on how to do this at
https://docs.trafficserver.apache.org/en/latest/admin-guide/plugins/cachekey.en.html
- Sudheer
> On Oct 27, 2016, at 12:22 AM, Randeep <ra...@gmail.com> wrote:
>
> Hi all,
>
> We are using ats to cache our live MPEG dash and HLS streams.
> We use version 6.2.0 with parent-child architecture.
>
> In the requests there are some query strings like timestamps for playing
> catchup,start-over,timeshift etc.
>
> Before the requests where not caching so we added @pparam=no-query-string
> so that it will cache without query stings. I thought it will strip the
> query strings only when caching but it is stripping the query-string while
> forwarding to domain also. Because of this a lot of services like catchup
> doesn't work.
>
> remap.config
> map http://jitp1.domain.com/ http://bglrcache1.domain.com/
> @plugin=/opt/ats/libexec/trafficserver/regex_remap.so
> @pparam=/opt/ats/etc/trafficserver/jitp1.config @pparam=no-query-string
>
> cat /opt/ats/etc/trafficserver/jitp5.config
> ^/(.*)$ http://bglrcache5.domain.com/$1
>
> How can we achieve both url forwarding with query sting and caching the
> content without the query-string?
>
> --
> Randeep
> Mob: +919447831699[kerala]
> Mob: +919880050349[B'lore]
> http://twitter.com/Randeeppr
> http://in.linkedin.com/in/randeeppr
>
>
> [image: --]
> Randeep Raman
> [image: http://]about.me/Randeeppr
> <http://about.me/Randeeppr>
Re: Query strings are not forwarded
Posted by Sudheer Vinukonda <su...@yahoo.com>.
You may try using cachekey plugin to ignore the query params only from the cache key and not from remap, instead of using regex_remap with no-query-string param, which will basically remove the query string from remap.
There is an excellent documentation and some examples on how to do this at
https://docs.trafficserver.apache.org/en/latest/admin-guide/plugins/cachekey.en.html
- Sudheer
> On Oct 27, 2016, at 12:22 AM, Randeep <ra...@gmail.com> wrote:
>
> Hi all,
>
> We are using ats to cache our live MPEG dash and HLS streams.
> We use version 6.2.0 with parent-child architecture.
>
> In the requests there are some query strings like timestamps for playing
> catchup,start-over,timeshift etc.
>
> Before the requests where not caching so we added @pparam=no-query-string
> so that it will cache without query stings. I thought it will strip the
> query strings only when caching but it is stripping the query-string while
> forwarding to domain also. Because of this a lot of services like catchup
> doesn't work.
>
> remap.config
> map http://jitp1.domain.com/ http://bglrcache1.domain.com/
> @plugin=/opt/ats/libexec/trafficserver/regex_remap.so
> @pparam=/opt/ats/etc/trafficserver/jitp1.config @pparam=no-query-string
>
> cat /opt/ats/etc/trafficserver/jitp5.config
> ^/(.*)$ http://bglrcache5.domain.com/$1
>
> How can we achieve both url forwarding with query sting and caching the
> content without the query-string?
>
> --
> Randeep
> Mob: +919447831699[kerala]
> Mob: +919880050349[B'lore]
> http://twitter.com/Randeeppr
> http://in.linkedin.com/in/randeeppr
>
>
> [image: --]
> Randeep Raman
> [image: http://]about.me/Randeeppr
> <http://about.me/Randeeppr>