You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Patrick O'Brien <pa...@tetrisblocks.net> on 2016/11/16 16:54:49 UTC
Using ATS with the s3_auth plugin
Hello,
We're currently evaluating ATS for our infrastructure and I have a couple of
implementation questions. First of all, what we are trying to do is use ATS
as a forward proxy for our internal servers in order to cache some S3 data
that all of the servers are currently pulling directly from S3. The more data
centers we add the worse and worse this operation gets (in terms of time and
cost).
Looking at the s3_auth plugin[0] documentation page it looks like the more
recent v4 auth mechanism is not yet supported, although the document hasn't
been updated since at least October 2015, but there have been changes to the
plugin since then[1] (including a change that adds a `version` option). Is
this plugin still stuck on AWS' v2 auth mechanism? I see the note in the
plugin itself about being only compatible with v2 has been removed[1],
although it looks like it does a check to make sure version is set to 2[2].
I tried to get the plugin working according to the docs and plugin source,
but every time we start ATS with the plugin defined we get the following
fatal error:
FATAL: unable to load remap.config
We have tried a few different remap.config entries, including just specifying
a config file, but no combo seemed to work. Here is an example entry:
map http://thetestbucket-cache.s3.amazonaws.com/
https://thetestbucket-cache.s3.amazonaws.com \
@plugin=s3_auth \
@pparam=--access_key @pparam=my-key \
@pparam=--secret_key @pparam=my-secret \
@pparam=--version @pparam=2
We have also tried just using a config file:
map http://thetestbucket-cache.s3.amazonaws.com/
https://thetestbucket-cache.s3.amazonaws.com \
@plugin=s3.auth \
@pparam=--config @pparam=/path/to/s3.config
We have also tried using "@plugin=s3_auth.so"
Here are some diagnostics I tried to run:
$ traffic_server -T"s3_auth"
traffic_server: using root directory
'/usr/local/Cellar/trafficserver/HEAD-3bc3beb'
[Nov 16 09:34:33.188] Server {0x7fffbf0723c0} DIAG: (s3_auth) plugin
is successfully initialized
FATAL: unable to load remap.config
traffic_server -C verify_config -D /usr/local/etc/trafficserver
traffic_server: using root directory
'/usr/local/Cellar/trafficserver/HEAD-3bc3beb'
[Nov 16 09:33:14.311] Server {0x7fffbf0723c0} DEBUG: <DNS.cc:1604
(ink_dns_init)> (dns) ink_dns_init: called with init_called = 0
NOTE: VERIFY
NOTE: VERIFY config dir: /usr/local/etc/trafficserver...
ERROR: Failed to load remap.config, exitStatus 1
...snip...
I would be more than happy to get a documentation PR going once I get this
all figured out, but in the meantime any advice on getting this going would
be fantastic. I can't seem to figure out what I am doing wrong in remap.config.
-patrick
Test OS: MacOS Sierra (10.12.1)
ATS Version: HEAD from git
0 - https://docs.trafficserver.apache.org/en/latest/admin-guide/plugins/s3_auth.en.html
1 - https://github.com/apache/trafficserver/commit/1c948c38045f41dd69ca2e1356b96dc0135f4f00#diff-7a7ff2985b971651587fb057834b1414R262
2 - https://github.com/apache/trafficserver/commit/1c948c38045f41dd69ca2e1356b96dc0135f4f00#diff-7a7ff2985b971651587fb057834b1414R68
Re: Using ATS with the s3_auth plugin
Posted by Sudheer Vinukonda <su...@yahoo.com>.
I think the @plugin param should be the name of the shared library (e.g. s3_auth.so) for the plugin (possibly along with the path if it's stored not in the standard plugin for).
> On Nov 16, 2016, at 8:54 AM, Patrick O'Brien <pa...@tetrisblocks.net> wrote:
>
> Hello,
>
> We're currently evaluating ATS for our infrastructure and I have a couple of
> implementation questions. First of all, what we are trying to do is use ATS
> as a forward proxy for our internal servers in order to cache some S3 data
> that all of the servers are currently pulling directly from S3. The more data
> centers we add the worse and worse this operation gets (in terms of time and
> cost).
>
> Looking at the s3_auth plugin[0] documentation page it looks like the more
> recent v4 auth mechanism is not yet supported, although the document hasn't
> been updated since at least October 2015, but there have been changes to the
> plugin since then[1] (including a change that adds a `version` option). Is
> this plugin still stuck on AWS' v2 auth mechanism? I see the note in the
> plugin itself about being only compatible with v2 has been removed[1],
> although it looks like it does a check to make sure version is set to 2[2].
>
> I tried to get the plugin working according to the docs and plugin source,
> but every time we start ATS with the plugin defined we get the following
> fatal error:
>
> FATAL: unable to load remap.config
>
> We have tried a few different remap.config entries, including just specifying
> a config file, but no combo seemed to work. Here is an example entry:
>
> map http://thetestbucket-cache.s3.amazonaws.com/
> https://thetestbucket-cache.s3.amazonaws.com \
> @plugin=s3_auth \
> @pparam=--access_key @pparam=my-key \
> @pparam=--secret_key @pparam=my-secret \
> @pparam=--version @pparam=2
>
> We have also tried just using a config file:
>
> map http://thetestbucket-cache.s3.amazonaws.com/
> https://thetestbucket-cache.s3.amazonaws.com \
> @plugin=s3.auth \
> @pparam=--config @pparam=/path/to/s3.config
>
> We have also tried using "@plugin=s3_auth.so"
>
> Here are some diagnostics I tried to run:
>
> $ traffic_server -T"s3_auth"
> traffic_server: using root directory
> '/usr/local/Cellar/trafficserver/HEAD-3bc3beb'
> [Nov 16 09:34:33.188] Server {0x7fffbf0723c0} DIAG: (s3_auth) plugin
> is successfully initialized
> FATAL: unable to load remap.config
>
> traffic_server -C verify_config -D /usr/local/etc/trafficserver
> traffic_server: using root directory
> '/usr/local/Cellar/trafficserver/HEAD-3bc3beb'
> [Nov 16 09:33:14.311] Server {0x7fffbf0723c0} DEBUG: <DNS.cc:1604
> (ink_dns_init)> (dns) ink_dns_init: called with init_called = 0
> NOTE: VERIFY
>
> NOTE: VERIFY config dir: /usr/local/etc/trafficserver...
>
> ERROR: Failed to load remap.config, exitStatus 1
>
> ...snip...
>
> I would be more than happy to get a documentation PR going once I get this
> all figured out, but in the meantime any advice on getting this going would
> be fantastic. I can't seem to figure out what I am doing wrong in remap.config.
>
> -patrick
>
> Test OS: MacOS Sierra (10.12.1)
> ATS Version: HEAD from git
>
> 0 - https://docs.trafficserver.apache.org/en/latest/admin-guide/plugins/s3_auth.en.html
> 1 - https://github.com/apache/trafficserver/commit/1c948c38045f41dd69ca2e1356b96dc0135f4f00#diff-7a7ff2985b971651587fb057834b1414R262
> 2 - https://github.com/apache/trafficserver/commit/1c948c38045f41dd69ca2e1356b96dc0135f4f00#diff-7a7ff2985b971651587fb057834b1414R68