You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by "Sandeep Kalidindi at PaGaLGuY.com" <sa...@pagalguy.com> on 2010/05/26 07:39:18 UTC
Connecting to Database
Hi all,
i am about to write a couple of transformation plugins to ATS
which will fetch data from the database so that all the read requests will
be taken care of by the ATS itself while write requests are passed on to the
backend servers.
was checking the architecture and it looks fine by me. wanted to ask the
community if there is any major why we should not connect to the database
from ATS before i spend considerable time on it.
Cheers,
Deepu.
Re: Connecting to Database
Posted by "Sandeep Kalidindi at PaGaLGuY.com" <sa...@pagalguy.com>.
Thanks a lot for pointing me in the right direction.
Seems like it is a bit dangerous to use blocking calls. i will play around
and will figure things.
Thanks a lot again for your help guys.
Cheers,
Deepu.
On Thu, May 27, 2010 at 12:05 AM, John Plevyak <jp...@apache.org> wrote:
> Just an FYI: you don't have to involve HTTP at all.
>
> You can use INKNetConnect, INKConnReadVIOGet, INKVConnReed
> and the corresponding write functions to talk directly to the
> SQL server.
>
> You can then use a general SQL parser and generator (one not
> tied to I/O). You can also use the API to read and write to
> the cache INKCacheRead INKCacheWrite.
>
> If you are building an HTTP->SQL gateway, you can also use the
> API functions for parsing and handling HTTP headers w/o using
> the HTTP engine or a transform plugin.
> john
>
> On 5/26/2010 11:09 AM, Steve Jiang wrote:
> > Hi Sandeep,
> >
> > A transformation plugin can accomplish what you want, but you will have
> > to setup a "dummy" HTTP origin server to provide a fake response so that
> > ATS has something to transform.
> >
> > If you want ATS to handle the request itself without connecting to an
> > origin HTTP server, you can use INKHttpTxnIntercept[1] to intercept
> > requests that you detect can be served from ATS. If you choose to go
> > this route, you'll have to build the response HTTP headers as well as
> > the body to the client VConnection.
> >
> > As far as connecting to the database, keep in mind that if a blocking
> > library is used database requests will block other connections on the
> > same thread. Using INKNetConnect with writing the protocol directly
> > will allow you to talk to the DB asynchronously and continue handling
> > other requests.
> >
> > --Steve
> >
> > [1]
> >
> http://trafficserver.apache.org/docs/v2/sdk/InterceptingHTTPTransactionFuncs.html
> >
> >
> > Sandeep Kalidindi at PaGaLGuY.com wrote, on 5/25/10 10:39 PM:
> >> Hi all,
> >> i am about to write a couple of transformation plugins to
> >> ATS which will fetch data from the database so that all the read
> >> requests will be taken care of by the ATS itself while write requests
> >> are passed on to the backend servers.
> >> was checking the architecture and it looks fine by me. wanted to ask
> >> the community if there is any major why we should not connect to the
> >> database from ATS before i spend considerable time on it.
> >>
> >> Cheers,
> >> Deepu.
>
>
Re: Connecting to Database
Posted by John Plevyak <jp...@apache.org>.
Just an FYI: you don't have to involve HTTP at all.
You can use INKNetConnect, INKConnReadVIOGet, INKVConnReed
and the corresponding write functions to talk directly to the
SQL server.
You can then use a general SQL parser and generator (one not
tied to I/O). You can also use the API to read and write to
the cache INKCacheRead INKCacheWrite.
If you are building an HTTP->SQL gateway, you can also use the
API functions for parsing and handling HTTP headers w/o using
the HTTP engine or a transform plugin.
john
On 5/26/2010 11:09 AM, Steve Jiang wrote:
> Hi Sandeep,
>
> A transformation plugin can accomplish what you want, but you will have
> to setup a "dummy" HTTP origin server to provide a fake response so that
> ATS has something to transform.
>
> If you want ATS to handle the request itself without connecting to an
> origin HTTP server, you can use INKHttpTxnIntercept[1] to intercept
> requests that you detect can be served from ATS. If you choose to go
> this route, you'll have to build the response HTTP headers as well as
> the body to the client VConnection.
>
> As far as connecting to the database, keep in mind that if a blocking
> library is used database requests will block other connections on the
> same thread. Using INKNetConnect with writing the protocol directly
> will allow you to talk to the DB asynchronously and continue handling
> other requests.
>
> --Steve
>
> [1]
> http://trafficserver.apache.org/docs/v2/sdk/InterceptingHTTPTransactionFuncs.html
>
>
> Sandeep Kalidindi at PaGaLGuY.com wrote, on 5/25/10 10:39 PM:
>> Hi all,
>> i am about to write a couple of transformation plugins to
>> ATS which will fetch data from the database so that all the read
>> requests will be taken care of by the ATS itself while write requests
>> are passed on to the backend servers.
>> was checking the architecture and it looks fine by me. wanted to ask
>> the community if there is any major why we should not connect to the
>> database from ATS before i spend considerable time on it.
>>
>> Cheers,
>> Deepu.
Re: Connecting to Database
Posted by Steve Jiang <sj...@apache.org>.
Hi Sandeep,
A transformation plugin can accomplish what you want, but you will have
to setup a "dummy" HTTP origin server to provide a fake response so that
ATS has something to transform.
If you want ATS to handle the request itself without connecting to an
origin HTTP server, you can use INKHttpTxnIntercept[1] to intercept
requests that you detect can be served from ATS. If you choose to go
this route, you'll have to build the response HTTP headers as well as
the body to the client VConnection.
As far as connecting to the database, keep in mind that if a blocking
library is used database requests will block other connections on the
same thread. Using INKNetConnect with writing the protocol directly
will allow you to talk to the DB asynchronously and continue handling
other requests.
--Steve
[1]
http://trafficserver.apache.org/docs/v2/sdk/InterceptingHTTPTransactionFuncs.html
Sandeep Kalidindi at PaGaLGuY.com wrote, on 5/25/10 10:39 PM:
> Hi all,
> i am about to write a couple of transformation plugins to ATS
> which will fetch data from the database so that all the read requests
> will be taken care of by the ATS itself while write requests are passed
> on to the backend servers.
>
> was checking the architecture and it looks fine by me. wanted to ask the
> community if there is any major why we should not connect to the
> database from ATS before i spend considerable time on it.
>
> Cheers,
> Deepu.
Re: Connecting to Database
Posted by John Plevyak <jp...@apache.org>.
INKNetConnect is a generic connect, it is not HTTP specific
(despite the comments).
You can use it for any protocol although as Sean says you
will have to do that at the byte stream level.
However, given that SQL is text, that shouldn't be that hard :)
john
On 5/26/2010 8:44 AM, Sean Cosgrave wrote:
> Hi Sandeep,
> There is currently no built-in support for making, say, MySQL calls from
> inside a plugin, but that does not prevent you from using another library
> inside your plugin (like mysql++) to make database calls.
>
> -Sean
>
>
> On 5/26/10 2:25 AM, "Sandeep Kalidindi at PaGaLGuY.com"
> <sa...@pagalguy.com> wrote:
>
>> From the docs i have a feeling ATS cannot make any non-http request to any
>> host . Is that so ??? Which means ATS plugins have no chance of connecting to
>> databases directly....
>>
>>
>>
>> INKNetConnect (contp, 127.0.0.1, 9999)
>>
>> the above call is the only call i saw to make network requests but the fact
>> that it doesn't take any protocol means i assume it only uses HTTP protocol.
>>
>> Please let me know if my understanding is wrong.
>>
>> Cheers,
>> Deepu.
>>
>>
>> On Wed, May 26, 2010 at 11:09 AM, Sandeep Kalidindi at PaGaLGuY.com
>> <sa...@pagalguy.com> wrote:
>>> Hi all,
>>> i am about to write a couple of transformation plugins to ATS
>>> which will fetch data from the database so that all the read requests will be
>>> taken care of by the ATS itself while write requests are passed on to the
>>> backend servers.
>>>
>>> was checking the architecture and it looks fine by me. wanted to ask the
>>> community if there is any major why we should not connect to the database
>>> from ATS before i spend considerable time on it.
>>>
>>> Cheers,
>>> Deepu.
>>
>>
>
>
Re: Connecting to Database
Posted by Sean Cosgrave <st...@yahoo-inc.com>.
Hi Sandeep,
There is currently no built-in support for making, say, MySQL calls from
inside a plugin, but that does not prevent you from using another library
inside your plugin (like mysql++) to make database calls.
-Sean
On 5/26/10 2:25 AM, "Sandeep Kalidindi at PaGaLGuY.com"
<sa...@pagalguy.com> wrote:
> From the docs i have a feeling ATS cannot make any non-http request to any
> host . Is that so ??? Which means ATS plugins have no chance of connecting to
> databases directly....
>
>
>
> INKNetConnect (contp, 127.0.0.1, 9999)
>
> the above call is the only call i saw to make network requests but the fact
> that it doesn't take any protocol means i assume it only uses HTTP protocol.
>
> Please let me know if my understanding is wrong.
>
> Cheers,
> Deepu.
>
>
> On Wed, May 26, 2010 at 11:09 AM, Sandeep Kalidindi at PaGaLGuY.com
> <sa...@pagalguy.com> wrote:
>> Hi all,
>> i am about to write a couple of transformation plugins to ATS
>> which will fetch data from the database so that all the read requests will be
>> taken care of by the ATS itself while write requests are passed on to the
>> backend servers.
>>
>> was checking the architecture and it looks fine by me. wanted to ask the
>> community if there is any major why we should not connect to the database
>> from ATS before i spend considerable time on it.
>>
>> Cheers,
>> Deepu.
>
>
Re: Connecting to Database
Posted by "Sandeep Kalidindi at PaGaLGuY.com" <sa...@pagalguy.com>.
>From the docs i have a feeling ATS cannot make any non-http request to any
host . Is that so ??? Which means ATS plugins have no chance of connecting
to databases directly....
INKNetConnect (contp, 127.0.0.1, 9999)
the above call is the only call i saw to make network requests but the fact
that it doesn't take any protocol means i assume it only uses HTTP protocol.
Please let me know if my understanding is wrong.
Cheers,
Deepu.
On Wed, May 26, 2010 at 11:09 AM, Sandeep Kalidindi at PaGaLGuY.com <
sandeep.kalidindi@pagalguy.com> wrote:
> Hi all,
> i am about to write a couple of transformation plugins to ATS
> which will fetch data from the database so that all the read requests will
> be taken care of by the ATS itself while write requests are passed on to the
> backend servers.
>
> was checking the architecture and it looks fine by me. wanted to ask the
> community if there is any major why we should not connect to the database
> from ATS before i spend considerable time on it.
>
> Cheers,
> Deepu.
>