Plugin problems

[Joshua Tolley <eg...@gmail.com>]

A site I'm working with needs some fairly intelligent edge caching, and I've
been trying out traffic server to implement it. Enough has gone wrong that I'm
wondering if either I'm doing something very wrong, or traffic server really
isn't suited for the things I'm trying to do.

For instance, I set up traffic server as a reverse proxy, in front of a local
apache instance. I tried using cache.config to cache all png files and no html
files, like this:

url_regex=localhost suffix=png ttl-in-cache=10m
url_regex=localhost suffix=html action=never-cache

I stopped traffic server, deleted the cache file, restarted traffic server,
and requested an html file and a png image through traffic server including
a "Cache-control: only-if-cached" header to see if the files would come from
the cache. As expected, I got a 504 Not Cached error, as expected. I requested
the files again, without the header, and received them normally. At this point
I'm hoping only the image file is cached, but when I request the files again,
with the Cache-control header, I get both of them normally.

So then I started fiddling with some of the example plugins. I managed to
compile the add-header and output-headers plugins easily enough, and to
install them. But the add-header plugin didn't add any headers, and the
output-headers plugin didn't output any headers. I added some INKDebug() lines
to both plugins and recompiled them so I could be sure they were running, and
saw all kinds of output telling me, for instance, the add-header plugin was
running and trying to add a header. But no header showed up on the output.

Finally, a more general question. The documentation says plugins can't modify
the cache directly. Some of our requirements are that we need to cache or not
cache objects based on headers, cookies, etc., but I'm not entirely sure
that's actually possible. I'd appreciate any comments you can give. Thanks in
advance.

--
Joshua Tolley

Re: Plugin problems

[Leif Hedstrom <zw...@apache.org>]

On 02/22/2011 05:12 PM, Joshua Tolley wrote:
>
> That does, and is kinda what I was thinking we'd have to do. For instance,
> some of our business logic said, "Given the presence of various
> application-specific headers or cookies, use the cache where normally we
> wouldn't, or alternatively, don't use it where we typically would." I assume
> the technique in this case would be to modify headers at some point in the
> process such that TS would decide to cache the response on its own?


Right. We have some "canned" configuration options available in 
records.config and cache.config, to do some of this, but specific rules 
like you describe more likely than not will require a custom C/C++ 
plugin (at least until we finish the LUA plugin, and it shows reasonable 
performance).

-- leif

Re: Plugin problems

[Joshua Tolley <eg...@gmail.com>]

On Tue, Feb 22, 2011 at 04:11:54PM -0700, Leif Hedstrom wrote:
> sorry for the late reply, hopefully you're still looking for an answer  
> to get going with Apache TS :).

To be honest, those leading the project in question have pretty much decided
to go with another option, however I'm still interested in knowing what's
going wrong, in particular for possible use of TS with other projects.

>> For instance, I set up traffic server as a reverse proxy, in front of a local
>> apache instance. I tried using cache.config to cache all png files and no html
>> files, like this:
>>
>> url_regex=localhost suffix=png ttl-in-cache=10m
>> url_regex=localhost suffix=html action=never-cache
>
> I suspect this is a bug, it seems we do not honor / use the suffix "mod"  
> properly. I've filed a bug (TS-674) to track this.

I'm glad it's not obviously my fault :)

> Hmm, the "example" headers generally are intended as "examples" only,  
> i.e. not for production use. As such, they ought to still use, but no  
> one really is testing them afaik. I've made a slightly better example  
> plugin for headers manipulation available on the SVN server, in
>
>     http://svn.apache.org/repos/asf/trafficserver/plugins/header_filter/
>
>
> Note that this plugin requires current trunk to compile as of a few days  
> ago.

I'll have to check it out. I realized the example plugins didn't really do
anything a production system might actually want, but I wasn't able to get
them to do anything, really.

>> Finally, a more general question. The documentation says plugins can't modify
>> the cache directly. Some of our requirements are that we need to cache or not
>> cache objects based on headers, cookies, etc., but I'm not entirely sure
>> that's actually possible. I'd appreciate any comments you can give. Thanks in
>> advance.
>
> You can't (easily) modify HTTP objects once they are in cache. But you  
> can modify objects before they get in cache, or modify the responses so  
> that they don't end up in cache etc. I hope that makes any sense?

That does, and is kinda what I was thinking we'd have to do. For instance,
some of our business logic said, "Given the presence of various
application-specific headers or cookies, use the cache where normally we
wouldn't, or alternatively, don't use it where we typically would." I assume
the technique in this case would be to modify headers at some point in the
process such that TS would decide to cache the response on its own?

- Josh

Re: Plugin problems

[Leif Hedstrom <zw...@apache.org>]

On 02/15/2011 05:09 PM, Joshua Tolley wrote:
> A site I'm working with needs some fairly intelligent edge caching, and I've
> been trying out traffic server to implement it. Enough has gone wrong that I'm
> wondering if either I'm doing something very wrong, or traffic server really
> isn't suited for the things I'm trying to do.

Hi Joshua,

sorry for the late reply, hopefully you're still looking for an answer 
to get going with Apache TS :).

> For instance, I set up traffic server as a reverse proxy, in front of a local
> apache instance. I tried using cache.config to cache all png files and no html
> files, like this:
>
> url_regex=localhost suffix=png ttl-in-cache=10m
> url_regex=localhost suffix=html action=never-cache

I suspect this is a bug, it seems we do not honor / use the suffix "mod" 
properly. I've filed a bug (TS-674) to track this.


> So then I started fiddling with some of the example plugins. I managed to
> compile the add-header and output-headers plugins easily enough, and to
> install them. But the add-header plugin didn't add any headers, and the
> output-headers plugin didn't output any headers. I added some INKDebug() lines
> to both plugins and recompiled them so I could be sure they were running, and
> saw all kinds of output telling me, for instance, the add-header plugin was
> running and trying to add a header. But no header showed up on the output.
Hmm, the "example" headers generally are intended as "examples" only, 
i.e. not for production use. As such, they ought to still use, but no 
one really is testing them afaik. I've made a slightly better example 
plugin for headers manipulation available on the SVN server, in

     http://svn.apache.org/repos/asf/trafficserver/plugins/header_filter/


Note that this plugin requires current trunk to compile as of a few days 
ago.

> Finally, a more general question. The documentation says plugins can't modify
> the cache directly. Some of our requirements are that we need to cache or not
> cache objects based on headers, cookies, etc., but I'm not entirely sure
> that's actually possible. I'd appreciate any comments you can give. Thanks in
> advance.


You can't (easily) modify HTTP objects once they are in cache. But you 
can modify objects before they get in cache, or modify the responses so 
that they don't end up in cache etc. I hope that makes any sense?

Cheers,

-- leif