You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by Victor Lucero <vi...@ipartner.cl> on 2010/12/09 04:36:33 UTC
Catch and modify OS conection
Hi.
Im writing a plugin to implement Load balancing. im facing some problem doing it with a remap plugin because its looks like when i alter new_host its also alter the "host" header in the http petition (and i dont want that)
I need a way to alter the conection to Original Server and change the ip(so i can select between the servers in the pool). but there is no hook in this step.
Atte. Victor Lucero.
Re: Catch and modify OS conection
Posted by Victor Lucero <vi...@ipartner.cl>.
looking thought the code i found this. (HttpSM:4197)
Action *pvc_action_handle = t->connect_re(this);
the values like t_state.current.server->name t_state.current.server->ip
got what i need . so now i need a way to configure this elements and do some scheduled health check, there is a way to add or implement new hooks for this step and some functions to the api, i mean, how i can implement a new hook and a new function to edit t_state w/o altering much the source code.(still looking how implement new functions/hooks in the api) but i got a long way to go searching thought the source code :S
-- Victor Lucero.
Api developer Wannabe.
El 09-12-2010, a las 10:28, Victor Lucero escribió:
> may i suggest just add a new hook before OS conection (post cache miss and previous to pick address (http://trafficserver.apache.org/docs/v2/sdk/HTTPHooksAndTransactions.html)
>
> if there's no health servers just send to the first (so it will fail)or just send back a error.
>
> in another part of the plugin i just add a schedule to test the servers health.
> and every "pick address" just need to follow a algorithm between the server pool. (and the cache is just going to get the content of the url he need w/o noticing the origin of the content)
>
>
> in this way if i get a hook like this
>
>
> if (INKHttpHookAdd(TS_HTTP_PICK_ADDR, INKContCreate(pic_addr, NULL)) == INK_ERROR) {
>
> }
>
>
> static int pic_addr(INKCont contp, INKEvent event,void *edata);
>
>
>
> edata might contain the header data so i can just add a regex check to validate the hostname and select a server from the pool in the algorithm order.
>
>
> --Victor Lucero
>
> El 09-12-2010, a las 4:06, ming.zym@gmail.com escribió:
>
>> we are thinking of L7 service health checking in connection manager to
>> OS, still investing.
>>
>> if you want do full balance, that is hard, I think we have many
>> checkpoints:
>> 1, HostDB strict RR, proxy.config.hostdb.strict_round_robin, that may
>> make all HostDB response strict RR.
>> 2, need a strict RR connection manager to OS, I think that will need
>> some work. and may or may not be a design problem, as we are not
>> guarantee to connnect all OS hosts for now. and the full balance may or
>> may not make sense, it will depends on how many TS hosts, how many OS
>> hosts, and how many ts2os connection/request.
>>
>> I'd like to have it in a full proxy env.
>>
>> btw: TS-316 maybe relate to your requirement too.
>> https://issues.apache.org/jira/browse/TS-316
>>
>>
>> 在 2010-12-09四的 01:18 -0300，Victor Lucero写道：
>>> El 09-12-2010, a las 0:56, Leif Hedstrom escribió:
>>>
>>>> On 12/08/2010 08:45 PM, Leif Hedstrom wrote:
>>>>> On 12/08/2010 08:36 PM, Victor Lucero wrote:
>>>>>> Hi.
>>>>>> Im writing a plugin to implement Load balancing. im facing some problem doing it with a remap plugin because its looks like when i alter new_host its also alter the "host" header in the http petition (and i dont want that)
>>>>>> I need a way to alter the conection to Original Server and change the ip(so i can select between the servers in the pool). but there is no hook in this step.
>>>>>
>>>>>
>>>>> You have two options:
>>>>
>>>>
>>>> Well, there's a third option, I meant to say there are "two options" if you want to do this as a remap plugin (which makes a fair amount of sense, since you might want to easily configure this plugin per remap rule).
>>>>
>>>> The third option is to use "trunk", which has a few changes to the hooks and APIs. In particular, there are two useful hooks (one is merely renamed, but still):
>>>>
>>>> TS_HTTP_PRE_REMAP_HOOK
>>>> TS_HTTP_POST_REMAP_HOOK
>>>>
>>>>
>>>
>>> the problem in the remap plugin is that not every request end in OS and the balance will not get homogeneus.
>>>
>>>
>>>> In addition, there's a new, very useful API (when running as a reverse proxy only):
>>>>
>>>> tsapi TSReturnCode TSSkipRemappingSet(TSHttpTxn txnp, int flag);
>>>>
>>>>
>>>> All three of my "suggestions" probably suffers from the fact that changing the destination "url" also affects the cache key (if the content is cacheable). In most cases, that would be undesirable, so you should also look into using this API to control the cache key:
>>>>
>>>> tsapi TSReturnCode TSCacheUrlSet(TSHttpTxn txnp, const char *url, int length);
>>>>
>>>>
>>>> As a side note, for real load balancing, we need a way to intercept all DNS / hostDB lookups in a plugin. So, you'd do something like
>>>>
>>>> map http://www.example.com/ http://real.example.com/
>>>>
>>>>
>>>> and when real.example.com is looked up in our hostDB / DNS (on every request), you would intercept it with a plugin (new hook required, there's no way to do this now afaik) that allows you to return an IP based on your own code. It's unclear to me right now how such a hook would interact with hostDB, and the existing code we have for marking IPs as down etc. (i.e. it's important for the core to communicate connect failures to a plugin in a new hook like this). It's something we should noodle about (I'll file a bug later), it'd be great to get core changes in before v3.0.
>>>>
>>>> Cheers,
>>>>
>>>
>>> editing the response of the dns will do a better trick , also adding some health test code (in the plugin. nothing really complicated) and adding a hook near the connection to the OS for stats (also needed if i want to implement least weight algorithm.)
>>>
>>> thanks.
>>>
>>>> -- leif
>>>>
>>>
>>> -Victor Lucero.
>>
>>
>
Re: Catch and modify OS conection
Posted by Victor Lucero <vi...@ipartner.cl>.
may i suggest just add a new hook before OS conection (post cache miss and previous to pick address (http://trafficserver.apache.org/docs/v2/sdk/HTTPHooksAndTransactions.html)
if there's no health servers just send to the first (so it will fail)or just send back a error.
in another part of the plugin i just add a schedule to test the servers health.
and every "pick address" just need to follow a algorithm between the server pool. (and the cache is just going to get the content of the url he need w/o noticing the origin of the content)
in this way if i get a hook like this
if (INKHttpHookAdd(TS_HTTP_PICK_ADDR, INKContCreate(pic_addr, NULL)) == INK_ERROR) {
}
static int pic_addr(INKCont contp, INKEvent event,void *edata);
edata might contain the header data so i can just add a regex check to validate the hostname and select a server from the pool in the algorithm order.
--Victor Lucero
El 09-12-2010, a las 4:06, ming.zym@gmail.com escribió:
> we are thinking of L7 service health checking in connection manager to
> OS, still investing.
>
> if you want do full balance, that is hard, I think we have many
> checkpoints:
> 1, HostDB strict RR, proxy.config.hostdb.strict_round_robin, that may
> make all HostDB response strict RR.
> 2, need a strict RR connection manager to OS, I think that will need
> some work. and may or may not be a design problem, as we are not
> guarantee to connnect all OS hosts for now. and the full balance may or
> may not make sense, it will depends on how many TS hosts, how many OS
> hosts, and how many ts2os connection/request.
>
> I'd like to have it in a full proxy env.
>
> btw: TS-316 maybe relate to your requirement too.
> https://issues.apache.org/jira/browse/TS-316
>
>
> 在 2010-12-09四的 01:18 -0300，Victor Lucero写道：
>> El 09-12-2010, a las 0:56, Leif Hedstrom escribió:
>>
>>> On 12/08/2010 08:45 PM, Leif Hedstrom wrote:
>>>> On 12/08/2010 08:36 PM, Victor Lucero wrote:
>>>>> Hi.
>>>>> Im writing a plugin to implement Load balancing. im facing some problem doing it with a remap plugin because its looks like when i alter new_host its also alter the "host" header in the http petition (and i dont want that)
>>>>> I need a way to alter the conection to Original Server and change the ip(so i can select between the servers in the pool). but there is no hook in this step.
>>>>
>>>>
>>>> You have two options:
>>>
>>>
>>> Well, there's a third option, I meant to say there are "two options" if you want to do this as a remap plugin (which makes a fair amount of sense, since you might want to easily configure this plugin per remap rule).
>>>
>>> The third option is to use "trunk", which has a few changes to the hooks and APIs. In particular, there are two useful hooks (one is merely renamed, but still):
>>>
>>> TS_HTTP_PRE_REMAP_HOOK
>>> TS_HTTP_POST_REMAP_HOOK
>>>
>>>
>>
>> the problem in the remap plugin is that not every request end in OS and the balance will not get homogeneus.
>>
>>
>>> In addition, there's a new, very useful API (when running as a reverse proxy only):
>>>
>>> tsapi TSReturnCode TSSkipRemappingSet(TSHttpTxn txnp, int flag);
>>>
>>>
>>> All three of my "suggestions" probably suffers from the fact that changing the destination "url" also affects the cache key (if the content is cacheable). In most cases, that would be undesirable, so you should also look into using this API to control the cache key:
>>>
>>> tsapi TSReturnCode TSCacheUrlSet(TSHttpTxn txnp, const char *url, int length);
>>>
>>>
>>> As a side note, for real load balancing, we need a way to intercept all DNS / hostDB lookups in a plugin. So, you'd do something like
>>>
>>> map http://www.example.com/ http://real.example.com/
>>>
>>>
>>> and when real.example.com is looked up in our hostDB / DNS (on every request), you would intercept it with a plugin (new hook required, there's no way to do this now afaik) that allows you to return an IP based on your own code. It's unclear to me right now how such a hook would interact with hostDB, and the existing code we have for marking IPs as down etc. (i.e. it's important for the core to communicate connect failures to a plugin in a new hook like this). It's something we should noodle about (I'll file a bug later), it'd be great to get core changes in before v3.0.
>>>
>>> Cheers,
>>>
>>
>> editing the response of the dns will do a better trick , also adding some health test code (in the plugin. nothing really complicated) and adding a hook near the connection to the OS for stats (also needed if i want to implement least weight algorithm.)
>>
>> thanks.
>>
>>> -- leif
>>>
>>
>> -Victor Lucero.
>
>
Re: Catch and modify OS conection
Posted by Victor Lucero <vi...@ipartner.cl>.
already working in HostDB.cc figuring a way to manually manage the hosts in the rr table, but for a good and homogeneous way for LB i think it need to be implemented somewhere near HttpSM.cc(after we got a dns response or even w/o a dns response at all (like haproxy)).
(i know my english is bad. sorry about that)
--Victor Lucero
El 09-12-2010, a las 4:06, ming.zym@gmail.com escribió:
> we are thinking of L7 service health checking in connection manager to
> OS, still investing.
>
> if you want do full balance, that is hard, I think we have many
> checkpoints:
> 1, HostDB strict RR, proxy.config.hostdb.strict_round_robin, that may
> make all HostDB response strict RR.
> 2, need a strict RR connection manager to OS, I think that will need
> some work. and may or may not be a design problem, as we are not
> guarantee to connnect all OS hosts for now. and the full balance may or
> may not make sense, it will depends on how many TS hosts, how many OS
> hosts, and how many ts2os connection/request.
>
> I'd like to have it in a full proxy env.
>
> btw: TS-316 maybe relate to your requirement too.
> https://issues.apache.org/jira/browse/TS-316
>
>
> 在 2010-12-09四的 01:18 -0300，Victor Lucero写道：
>> El 09-12-2010, a las 0:56, Leif Hedstrom escribió:
>>
>>> On 12/08/2010 08:45 PM, Leif Hedstrom wrote:
>>>> On 12/08/2010 08:36 PM, Victor Lucero wrote:
>>>>> Hi.
>>>>> Im writing a plugin to implement Load balancing. im facing some problem doing it with a remap plugin because its looks like when i alter new_host its also alter the "host" header in the http petition (and i dont want that)
>>>>> I need a way to alter the conection to Original Server and change the ip(so i can select between the servers in the pool). but there is no hook in this step.
>>>>
>>>>
>>>> You have two options:
>>>
>>>
>>> Well, there's a third option, I meant to say there are "two options" if you want to do this as a remap plugin (which makes a fair amount of sense, since you might want to easily configure this plugin per remap rule).
>>>
>>> The third option is to use "trunk", which has a few changes to the hooks and APIs. In particular, there are two useful hooks (one is merely renamed, but still):
>>>
>>> TS_HTTP_PRE_REMAP_HOOK
>>> TS_HTTP_POST_REMAP_HOOK
>>>
>>>
>>
>> the problem in the remap plugin is that not every request end in OS and the balance will not get homogeneus.
>>
>>
>>> In addition, there's a new, very useful API (when running as a reverse proxy only):
>>>
>>> tsapi TSReturnCode TSSkipRemappingSet(TSHttpTxn txnp, int flag);
>>>
>>>
>>> All three of my "suggestions" probably suffers from the fact that changing the destination "url" also affects the cache key (if the content is cacheable). In most cases, that would be undesirable, so you should also look into using this API to control the cache key:
>>>
>>> tsapi TSReturnCode TSCacheUrlSet(TSHttpTxn txnp, const char *url, int length);
>>>
>>>
>>> As a side note, for real load balancing, we need a way to intercept all DNS / hostDB lookups in a plugin. So, you'd do something like
>>>
>>> map http://www.example.com/ http://real.example.com/
>>>
>>>
>>> and when real.example.com is looked up in our hostDB / DNS (on every request), you would intercept it with a plugin (new hook required, there's no way to do this now afaik) that allows you to return an IP based on your own code. It's unclear to me right now how such a hook would interact with hostDB, and the existing code we have for marking IPs as down etc. (i.e. it's important for the core to communicate connect failures to a plugin in a new hook like this). It's something we should noodle about (I'll file a bug later), it'd be great to get core changes in before v3.0.
>>>
>>> Cheers,
>>>
>>
>> editing the response of the dns will do a better trick , also adding some health test code (in the plugin. nothing really complicated) and adding a hook near the connection to the OS for stats (also needed if i want to implement least weight algorithm.)
>>
>> thanks.
>>
>>> -- leif
>>>
>>
>> -Victor Lucero.
>
>
Re: Catch and modify OS conection
Posted by "ming.zym@gmail.com" <mi...@gmail.com>.
we are thinking of L7 service health checking in connection manager to
OS, still investing.
if you want do full balance, that is hard, I think we have many
checkpoints:
1, HostDB strict RR, proxy.config.hostdb.strict_round_robin, that may
make all HostDB response strict RR.
2, need a strict RR connection manager to OS, I think that will need
some work. and may or may not be a design problem, as we are not
guarantee to connnect all OS hosts for now. and the full balance may or
may not make sense, it will depends on how many TS hosts, how many OS
hosts, and how many ts2os connection/request.
I'd like to have it in a full proxy env.
btw: TS-316 maybe relate to your requirement too.
https://issues.apache.org/jira/browse/TS-316
在 2010-12-09四的 01:18 -0300，Victor Lucero写道：
> El 09-12-2010, a las 0:56, Leif Hedstrom escribió:
>
> > On 12/08/2010 08:45 PM, Leif Hedstrom wrote:
> >> On 12/08/2010 08:36 PM, Victor Lucero wrote:
> >>> Hi.
> >>> Im writing a plugin to implement Load balancing. im facing some problem doing it with a remap plugin because its looks like when i alter new_host its also alter the "host" header in the http petition (and i dont want that)
> >>> I need a way to alter the conection to Original Server and change the ip(so i can select between the servers in the pool). but there is no hook in this step.
> >>
> >>
> >> You have two options:
> >
> >
> > Well, there's a third option, I meant to say there are "two options" if you want to do this as a remap plugin (which makes a fair amount of sense, since you might want to easily configure this plugin per remap rule).
> >
> > The third option is to use "trunk", which has a few changes to the hooks and APIs. In particular, there are two useful hooks (one is merely renamed, but still):
> >
> > TS_HTTP_PRE_REMAP_HOOK
> > TS_HTTP_POST_REMAP_HOOK
> >
> >
>
> the problem in the remap plugin is that not every request end in OS and the balance will not get homogeneus.
>
>
> > In addition, there's a new, very useful API (when running as a reverse proxy only):
> >
> > tsapi TSReturnCode TSSkipRemappingSet(TSHttpTxn txnp, int flag);
> >
> >
> > All three of my "suggestions" probably suffers from the fact that changing the destination "url" also affects the cache key (if the content is cacheable). In most cases, that would be undesirable, so you should also look into using this API to control the cache key:
> >
> > tsapi TSReturnCode TSCacheUrlSet(TSHttpTxn txnp, const char *url, int length);
> >
> >
> > As a side note, for real load balancing, we need a way to intercept all DNS / hostDB lookups in a plugin. So, you'd do something like
> >
> > map http://www.example.com/ http://real.example.com/
> >
> >
> > and when real.example.com is looked up in our hostDB / DNS (on every request), you would intercept it with a plugin (new hook required, there's no way to do this now afaik) that allows you to return an IP based on your own code. It's unclear to me right now how such a hook would interact with hostDB, and the existing code we have for marking IPs as down etc. (i.e. it's important for the core to communicate connect failures to a plugin in a new hook like this). It's something we should noodle about (I'll file a bug later), it'd be great to get core changes in before v3.0.
> >
> > Cheers,
> >
>
> editing the response of the dns will do a better trick , also adding some health test code (in the plugin. nothing really complicated) and adding a hook near the connection to the OS for stats (also needed if i want to implement least weight algorithm.)
>
> thanks.
>
> > -- leif
> >
>
> -Victor Lucero.
Re: Catch and modify OS conection
Posted by Victor Lucero <vi...@ipartner.cl>.
El 09-12-2010, a las 0:56, Leif Hedstrom escribió:
> On 12/08/2010 08:45 PM, Leif Hedstrom wrote:
>> On 12/08/2010 08:36 PM, Victor Lucero wrote:
>>> Hi.
>>> Im writing a plugin to implement Load balancing. im facing some problem doing it with a remap plugin because its looks like when i alter new_host its also alter the "host" header in the http petition (and i dont want that)
>>> I need a way to alter the conection to Original Server and change the ip(so i can select between the servers in the pool). but there is no hook in this step.
>>
>>
>> You have two options:
>
>
> Well, there's a third option, I meant to say there are "two options" if you want to do this as a remap plugin (which makes a fair amount of sense, since you might want to easily configure this plugin per remap rule).
>
> The third option is to use "trunk", which has a few changes to the hooks and APIs. In particular, there are two useful hooks (one is merely renamed, but still):
>
> TS_HTTP_PRE_REMAP_HOOK
> TS_HTTP_POST_REMAP_HOOK
>
>
the problem in the remap plugin is that not every request end in OS and the balance will not get homogeneus.
> In addition, there's a new, very useful API (when running as a reverse proxy only):
>
> tsapi TSReturnCode TSSkipRemappingSet(TSHttpTxn txnp, int flag);
>
>
> All three of my "suggestions" probably suffers from the fact that changing the destination "url" also affects the cache key (if the content is cacheable). In most cases, that would be undesirable, so you should also look into using this API to control the cache key:
>
> tsapi TSReturnCode TSCacheUrlSet(TSHttpTxn txnp, const char *url, int length);
>
>
> As a side note, for real load balancing, we need a way to intercept all DNS / hostDB lookups in a plugin. So, you'd do something like
>
> map http://www.example.com/ http://real.example.com/
>
>
> and when real.example.com is looked up in our hostDB / DNS (on every request), you would intercept it with a plugin (new hook required, there's no way to do this now afaik) that allows you to return an IP based on your own code. It's unclear to me right now how such a hook would interact with hostDB, and the existing code we have for marking IPs as down etc. (i.e. it's important for the core to communicate connect failures to a plugin in a new hook like this). It's something we should noodle about (I'll file a bug later), it'd be great to get core changes in before v3.0.
>
> Cheers,
>
editing the response of the dns will do a better trick , also adding some health test code (in the plugin. nothing really complicated) and adding a hook near the connection to the OS for stats (also needed if i want to implement least weight algorithm.)
thanks.
> -- leif
>
-Victor Lucero.
Re: Catch and modify OS conection
Posted by Leif Hedstrom <zw...@apache.org>.
On 12/08/2010 08:45 PM, Leif Hedstrom wrote:
> On 12/08/2010 08:36 PM, Victor Lucero wrote:
>> Hi.
>> Im writing a plugin to implement Load balancing. im facing some
>> problem doing it with a remap plugin because its looks like when i
>> alter new_host its also alter the "host" header in the http petition
>> (and i dont want that)
>> I need a way to alter the conection to Original Server and change the
>> ip(so i can select between the servers in the pool). but there is no
>> hook in this step.
>
>
> You have two options:
Well, there's a third option, I meant to say there are "two options" if
you want to do this as a remap plugin (which makes a fair amount of
sense, since you might want to easily configure this plugin per remap rule).
The third option is to use "trunk", which has a few changes to the hooks
and APIs. In particular, there are two useful hooks (one is merely
renamed, but still):
TS_HTTP_PRE_REMAP_HOOK
TS_HTTP_POST_REMAP_HOOK
In addition, there's a new, very useful API (when running as a reverse
proxy only):
tsapi TSReturnCode TSSkipRemappingSet(TSHttpTxn txnp, int flag);
All three of my "suggestions" probably suffers from the fact that
changing the destination "url" also affects the cache key (if the
content is cacheable). In most cases, that would be undesirable, so you
should also look into using this API to control the cache key:
tsapi TSReturnCode TSCacheUrlSet(TSHttpTxn txnp, const char *url,
int length);
As a side note, for real load balancing, we need a way to intercept all
DNS / hostDB lookups in a plugin. So, you'd do something like
map http://www.example.com/ http://real.example.com/
and when real.example.com is looked up in our hostDB / DNS (on every
request), you would intercept it with a plugin (new hook required,
there's no way to do this now afaik) that allows you to return an IP
based on your own code. It's unclear to me right now how such a hook
would interact with hostDB, and the existing code we have for marking
IPs as down etc. (i.e. it's important for the core to communicate
connect failures to a plugin in a new hook like this). It's something we
should noodle about (I'll file a bug later), it'd be great to get core
changes in before v3.0.
Cheers,
-- leif
Re: Catch and modify OS conection
Posted by Leif Hedstrom <zw...@apache.org>.
On 12/08/2010 08:36 PM, Victor Lucero wrote:
> Hi.
> Im writing a plugin to implement Load balancing. im facing some problem doing it with a remap plugin because its looks like when i alter new_host its also alter the "host" header in the http petition (and i dont want that)
> I need a way to alter the conection to Original Server and change the ip(so i can select between the servers in the pool). but there is no hook in this step.
You have two options:
1) Set the global config to preserve the host header:
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
(this is actually the new default on trunk, which will be in v2.1.5, so
beware if you depend on it being disabled).
2) On each remap rule, you can toggle the global value, e.g.
map http://www.example.com http://real.example.com
@pristine_host_hdr=1
There's also a bug filed to improve the remap plugin APIs (I think they
need a major overhaul), see https://issues.apache.org/jira/browse/TS-317
, and to some extent (I'll comment more on it later),
https://issues.apache.org/jira/browse/TS-588 .
I hope this helps?
-- Leif