You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Thom Brown <th...@linux.com> on 2010/09/02 23:43:30 UTC
[users@httpd] Proxy streaming
Hi all,
I'm wondering if someone could help here. Basically what I want to do
is provide a proxy for a video stream so that only 1 request goes to
the destination server, but will server many people using some sort of
caching.
My virtual host entry looks something like this:
<VirtualHost *:80>
ServerName myserver.name
CacheEnable disk /vid
SetEnv proxy-sendchunked 1
<Location /vid>
Options FollowSymLinks
Order allow,deny
Allow from all
</Location>
ProxyPass /vid/ http://www.destination.server/vid/
</VirtualHost>
My mem_cache settings are:
MCacheSize 25000000
MCacheMaxObjectCount 5
MCacheMinObjectSize 1
MCacheMaxObjectSize 30000000
MCacheMaxStreamingBuffer 300000
However, this results in each request producing an additional request
to the destination server.
What do I need to do?
Thanks
--
Thom Brown
Twitter: @darkixion
IRC (freenode): dark_ixion
Registered Linux user: #516935
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Re: Proxy streaming
Posted by Peter Janovsky <pe...@yahoo.com>.
Varnish is a great tool. You would have to pre-cache your content to minimize first byte response time for large objects as varnish waits for the entire response from the backend to be cached before streaming to client. If object is in cache response times are great. I know the varnish team was looking at implementing streaming while caching, not sure if this is complete.
On Sep 6, 2010, at 12:28, Eric Covener <co...@gmail.com> wrote:
> On Mon, Sep 6, 2010 at 11:42 AM, Thom Brown <th...@linux.com> wrote:
>> On 6 September 2010 16:10, Eric Covener <co...@gmail.com> wrote:
>>> On Mon, Sep 6, 2010 at 10:55 AM, Thom Brown <th...@linux.com> wrote:
>>>> On 6 September 2010 12:49, Eric Covener <co...@gmail.com> wrote:
>>>>> Is the response cacheable?
>>>>>
>>>>> http://httpd.apache.org/docs/current/caching.html
>>>>
>>>> Just checked and it appears the header was returning no-cache in
>>>> Cache-Control. Using CacheIgnoreNoLastMod now. It meets all other
>>>> criteria.
>>>>
>>>>>>> MCacheMaxStreamingBuffer 300000
>>>>>
>>>>> Is it larger than 300KB?
>>>>
>>>> I left it at 300KB because I didn't want a huge latency. As it's a
>>>> stream, it'll be indefinitately long. Have I misunderstood this
>>>> setting?
>>>
>>> I think so, as soon as the cache has seen 300KB of the response (over
>>> multiple chunks), it will decide the response on the whole is too big
>>> to be cached.
>>
>> By that logic, streams can't really be cached using mem_cache as it
>> will always hit the limit. Do I need to look at something else, like
>> Varnish?
>
> If you define a stream as an infinitely long response, then I don't
> think it can be stored in memory.
>
> --
> Eric Covener
> covener@gmail.com
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Re: Proxy streaming
Posted by Eric Covener <co...@gmail.com>.
On Mon, Sep 6, 2010 at 11:42 AM, Thom Brown <th...@linux.com> wrote:
> On 6 September 2010 16:10, Eric Covener <co...@gmail.com> wrote:
>> On Mon, Sep 6, 2010 at 10:55 AM, Thom Brown <th...@linux.com> wrote:
>>> On 6 September 2010 12:49, Eric Covener <co...@gmail.com> wrote:
>>>> Is the response cacheable?
>>>>
>>>> http://httpd.apache.org/docs/current/caching.html
>>>
>>> Just checked and it appears the header was returning no-cache in
>>> Cache-Control. Using CacheIgnoreNoLastMod now. It meets all other
>>> criteria.
>>>
>>>>>> MCacheMaxStreamingBuffer 300000
>>>>
>>>> Is it larger than 300KB?
>>>
>>> I left it at 300KB because I didn't want a huge latency. As it's a
>>> stream, it'll be indefinitately long. Have I misunderstood this
>>> setting?
>>
>> I think so, as soon as the cache has seen 300KB of the response (over
>> multiple chunks), it will decide the response on the whole is too big
>> to be cached.
>
> By that logic, streams can't really be cached using mem_cache as it
> will always hit the limit. Do I need to look at something else, like
> Varnish?
If you define a stream as an infinitely long response, then I don't
think it can be stored in memory.
--
Eric Covener
covener@gmail.com
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Re: Proxy streaming
Posted by Thom Brown <th...@linux.com>.
On 6 September 2010 16:10, Eric Covener <co...@gmail.com> wrote:
> On Mon, Sep 6, 2010 at 10:55 AM, Thom Brown <th...@linux.com> wrote:
>> On 6 September 2010 12:49, Eric Covener <co...@gmail.com> wrote:
>>> Is the response cacheable?
>>>
>>> http://httpd.apache.org/docs/current/caching.html
>>
>> Just checked and it appears the header was returning no-cache in
>> Cache-Control. Using CacheIgnoreNoLastMod now. It meets all other
>> criteria.
>>
>>>>> MCacheMaxStreamingBuffer 300000
>>>
>>> Is it larger than 300KB?
>>
>> I left it at 300KB because I didn't want a huge latency. As it's a
>> stream, it'll be indefinitately long. Have I misunderstood this
>> setting?
>
> I think so, as soon as the cache has seen 300KB of the response (over
> multiple chunks), it will decide the response on the whole is too big
> to be cached.
By that logic, streams can't really be cached using mem_cache as it
will always hit the limit. Do I need to look at something else, like
Varnish?
--
Thom Brown
Twitter: @darkixion
IRC (freenode): dark_ixion
Registered Linux user: #516935
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Re: Proxy streaming
Posted by Eric Covener <co...@gmail.com>.
On Mon, Sep 6, 2010 at 10:55 AM, Thom Brown <th...@linux.com> wrote:
> On 6 September 2010 12:49, Eric Covener <co...@gmail.com> wrote:
>> Is the response cacheable?
>>
>> http://httpd.apache.org/docs/current/caching.html
>
> Just checked and it appears the header was returning no-cache in
> Cache-Control. Using CacheIgnoreNoLastMod now. It meets all other
> criteria.
>
>>>> MCacheMaxStreamingBuffer 300000
>>
>> Is it larger than 300KB?
>
> I left it at 300KB because I didn't want a huge latency. As it's a
> stream, it'll be indefinitately long. Have I misunderstood this
> setting?
I think so, as soon as the cache has seen 300KB of the response (over
multiple chunks), it will decide the response on the whole is too big
to be cached.
--
Eric Covener
covener@gmail.com
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Re: Proxy streaming
Posted by Thom Brown <th...@linux.com>.
On 6 September 2010 12:49, Eric Covener <co...@gmail.com> wrote:
> Is the response cacheable?
>
> http://httpd.apache.org/docs/current/caching.html
Just checked and it appears the header was returning no-cache in
Cache-Control. Using CacheIgnoreNoLastMod now. It meets all other
criteria.
>>> MCacheMaxStreamingBuffer 300000
>
> Is it larger than 300KB?
I left it at 300KB because I didn't want a huge latency. As it's a
stream, it'll be indefinitately long. Have I misunderstood this
setting?
--
Thom Brown
Twitter: @darkixion
IRC (freenode): dark_ixion
Registered Linux user: #516935
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Re: Proxy streaming
Posted by Eric Covener <co...@gmail.com>.
Is the response cacheable?
http://httpd.apache.org/docs/current/caching.html
>> MCacheMaxStreamingBuffer 300000
Is it larger than 300KB?
--
Eric Covener
covener@gmail.com
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] Re: Proxy streaming
Posted by Thom Brown <th...@linux.com>.
On 2 September 2010 22:43, Thom Brown <th...@linux.com> wrote:
> Hi all,
>
> I'm wondering if someone could help here. Basically what I want to do
> is provide a proxy for a video stream so that only 1 request goes to
> the destination server, but will server many people using some sort of
> caching.
>
> My virtual host entry looks something like this:
>
> <VirtualHost *:80>
> ServerName myserver.name
>
> CacheEnable disk /vid
>
> SetEnv proxy-sendchunked 1
>
> <Location /vid>
> Options FollowSymLinks
> Order allow,deny
> Allow from all
> </Location>
>
> ProxyPass /vid/ http://www.destination.server/vid/
> </VirtualHost>
>
> My mem_cache settings are:
>
> MCacheSize 25000000
> MCacheMaxObjectCount 5
> MCacheMinObjectSize 1
> MCacheMaxObjectSize 30000000
> MCacheMaxStreamingBuffer 300000
>
> However, this results in each request producing an additional request
> to the destination server.
>
> What do I need to do?
So is this setup not really possible with Apache?
--
Thom Brown
Twitter: @darkixion
IRC (freenode): dark_ixion
Registered Linux user: #516935
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org