You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by Dk Jack <dn...@gmail.com> on 2017/09/26 02:31:09 UTC
ssl conn and http
Hi,
I am adding a hook to get access to the ssl handshake data. I am using
something like:
TSHttpHookAdd(TS_SSL_CERT_HOOK, cb_cert);
In the ssl callback, I am extracting some info from the ssl connection and
I'd like to save the info. I'd like to later retrieve this info when my
Http hook call back is invoked.
TSHttpHookAdd(TS_HTTP_SEND_REQUEST_HDR_HOOK, cont);
When my HTTP hook callback is called, I get TSHttpTxn. From HttpTxn how to
retrieve the saved ssl information? What is the correct way to make this
work? Any help is appreciated. Thanks.
Dk.
Re: ssl conn and http
Posted by Dk Jack <dn...@gmail.com>.
All,
I've created a PR to address issue #2380 (Missing ssl session close hook)
and
#2388 (Ability to add contextual data on (Net)VConnections). Would
appreciate
a review. The PR includes includes API documentation as well for #2388 and
hence would appreciate a review for the documentation as well. Thanks.
D.
https://github.com/apache/trafficserver/pull/2783
On Wed, Sep 27, 2017 at 4:42 AM, Alan Carroll <
solidwallofcode@oath.com.invalid> wrote:
> Unfortunately I don't know of a reliable way to do clean up, which is one
> reason I would think of a plugin maintained table which at least has the
> possibility of cleanup. This does point out a problem with the mentioned
> issue - if that is done there must be some reliable way to clean it up.
>
> I'm not an SSL expert but something dangerous you could try is to see if
> there is an openSSL hook you can grab to do the cleanup. The dangerous part
> is it is likely the ATS core already has a callback in place so you'd need
> to retrieve that and call it from your hook.
>
Re: ssl conn and http
Posted by Dk Jack <dn...@gmail.com>.
Never mind, I figured out the issue. Please ignore... thanks.
On Mon, Oct 9, 2017 at 5:13 PM, Dk Jack <dn...@gmail.com> wrote:
> Alan,
> I've been working on the issues we discussed. I have most of it working.
> I've added two things:
>
> - Allow plugins to add user data to conn.
> bool TSVConnSetUserData(TSVConn connp, const char* name, void* data);
> void* TSVConnGetUserData(TSVConn connp, const char* name);
>
> Did not bother with reserving since user can get the data using name.
>
> - Added close hook for SSLNetVConnection. I couldn't add it to lower levels
> because, the lower levels (vconn, netvconn) are called after the ssl
> data-structures
> are cleaned up and I wanted the plugin callback to be synchronously so
> that user
> data can be cleaned up.
>
> Things are working for the most part. I am getting a crash on the second
> ssl request. I suspect
> I am missing something. Would appreciate it if you or someone more
> familiar with the
> code can let me know what I am doing wrong.
>
> I've attached the diff (the diff is based on 6.2.1 code base).
>
> Thanks,
> Dk
>
> PS: Stack trace...
>
> 0 0x00000000004afd82 crash_logger_invoke(int, siginfo_t*, void*) + 0x82
> 1 0x00002afa38eec390 __restore_rt + (nil)
> 2 0x00002afa39bd2428 gsignal + 0x38
> 3 0x00002afa39bd402a abort + 0x16a
> 4 0x00002afa37ba0adf ink_fatal_va(char const*, __va_list_tag*) + 0x9f
> 5 0x00002afa37ba0b79 ink_fatal(char const*, ...) + 0x99
> 6 0x00002afa37b9ec35 _ink_assert + 0x15
> 7 0x00000000004ca26d INKContInternal::handle_event(int, void*) + 0x13d
> 8 0x0000000000780b27 SSLNetVConnection::do_io_close(int) + 0x147
> 9 0x00000000005a7636 Http1ClientSession::do_io_close(int) + 0x116
> 10 0x00000000005ab86f Http1ClientSession::state_keep_alive(int, void*)
> + 0x3f
> 11 0x00000000007adbb0 UnixNetVConnection::readSignalDone(int,
> NetHandler*) + 0x50
> 12 0x00000000007891e8 SSLNetVConnection::net_read_io(NetHandler*,
> EThread*) + 0x898
> 13 0x0000000000798654 NetHandler::mainNetEvent(int, Event*) + 0x214
> 14 0x00000000007e0f12 EThread::process_event(Event*, int) + 0x92
> 15 0x00000000007e1bc9 EThread::execute() + 0x639
> 16 0x00000000007e0995 spawn_thread_internal(void*) + 0x55
> 17 0x00002afa38ee26ba start_thread + 0xca
> 18 0x00002afa39ca43dd clone + 0x6d
> 19 0x0000000000000000 0x0 + 0x6d
>
>
> On Wed, Sep 27, 2017 at 6:19 PM, Alan Carroll <solidwallofcode@oath.com.
> invalid> wrote:
>
>> I don't think I'd do it for TLS_CLOSE, but VCONN_CLOSE would likely be
>> worth it. There's an issue on that, linked from the issue I mentioned
>> above. It's about the same thing - plugin wants to attach data during TLS
>> handshake for later retrieval at the HTTP level that can be cleaned up if
>> something goes wrong.
>>
>> On Wed, Sep 27, 2017 at 4:29 PM, Dk Jack <dn...@gmail.com> wrote:
>>
>> > Yeah, I was thinking along similar lines...
>> >
>> > Do you think, it'd be a worth while effort to add a ssl-close event
>> hook?
>> >
>> > On Wed, Sep 27, 2017 at 1:31 PM, Alan Carroll <
>> > solidwallofcode@oath.com.invalid> wrote:
>> >
>> > > I'd probably put a time stamp in and update it during the hook
>> > processing.
>> > > If you waited 5 minutes or so before clearing, that should be enough.
>> If
>> > > it's been 5 minutes since the last activity the connection should be
>> > dead,
>> > > unless you're shipping big files.
>> > >
>> > > Or, alternatively, in READ_REQUEST_HDR_HOOK pull the data out and
>> attach
>> > it
>> > > to the txn, cleaning it up in TXN_CLOSE. Set a time stamp on entry in
>> the
>> > > table and if that's been 5 minutes, the connection has definitely
>> timed
>> > > out.
>> > >
>> >
>>
>
>
Re: ssl conn and http
Posted by Dk Jack <dn...@gmail.com>.
Alan,
I've been working on the issues we discussed. I have most of it working.
I've added two things:
- Allow plugins to add user data to conn.
bool TSVConnSetUserData(TSVConn connp, const char* name, void* data);
void* TSVConnGetUserData(TSVConn connp, const char* name);
Did not bother with reserving since user can get the data using name.
- Added close hook for SSLNetVConnection. I couldn't add it to lower levels
because, the lower levels (vconn, netvconn) are called after the ssl
data-structures
are cleaned up and I wanted the plugin callback to be synchronously so
that user
data can be cleaned up.
Things are working for the most part. I am getting a crash on the second
ssl request. I suspect
I am missing something. Would appreciate it if you or someone more familiar
with the
code can let me know what I am doing wrong.
I've attached the diff (the diff is based on 6.2.1 code base).
Thanks,
Dk
PS: Stack trace...
0 0x00000000004afd82 crash_logger_invoke(int, siginfo_t*, void*) + 0x82
1 0x00002afa38eec390 __restore_rt + (nil)
2 0x00002afa39bd2428 gsignal + 0x38
3 0x00002afa39bd402a abort + 0x16a
4 0x00002afa37ba0adf ink_fatal_va(char const*, __va_list_tag*) + 0x9f
5 0x00002afa37ba0b79 ink_fatal(char const*, ...) + 0x99
6 0x00002afa37b9ec35 _ink_assert + 0x15
7 0x00000000004ca26d INKContInternal::handle_event(int, void*) + 0x13d
8 0x0000000000780b27 SSLNetVConnection::do_io_close(int) + 0x147
9 0x00000000005a7636 Http1ClientSession::do_io_close(int) + 0x116
10 0x00000000005ab86f Http1ClientSession::state_keep_alive(int, void*) +
0x3f
11 0x00000000007adbb0 UnixNetVConnection::readSignalDone(int,
NetHandler*) + 0x50
12 0x00000000007891e8 SSLNetVConnection::net_read_io(NetHandler*,
EThread*) + 0x898
13 0x0000000000798654 NetHandler::mainNetEvent(int, Event*) + 0x214
14 0x00000000007e0f12 EThread::process_event(Event*, int) + 0x92
15 0x00000000007e1bc9 EThread::execute() + 0x639
16 0x00000000007e0995 spawn_thread_internal(void*) + 0x55
17 0x00002afa38ee26ba start_thread + 0xca
18 0x00002afa39ca43dd clone + 0x6d
19 0x0000000000000000 0x0 + 0x6d
On Wed, Sep 27, 2017 at 6:19 PM, Alan Carroll <
solidwallofcode@oath.com.invalid> wrote:
> I don't think I'd do it for TLS_CLOSE, but VCONN_CLOSE would likely be
> worth it. There's an issue on that, linked from the issue I mentioned
> above. It's about the same thing - plugin wants to attach data during TLS
> handshake for later retrieval at the HTTP level that can be cleaned up if
> something goes wrong.
>
> On Wed, Sep 27, 2017 at 4:29 PM, Dk Jack <dn...@gmail.com> wrote:
>
> > Yeah, I was thinking along similar lines...
> >
> > Do you think, it'd be a worth while effort to add a ssl-close event hook?
> >
> > On Wed, Sep 27, 2017 at 1:31 PM, Alan Carroll <
> > solidwallofcode@oath.com.invalid> wrote:
> >
> > > I'd probably put a time stamp in and update it during the hook
> > processing.
> > > If you waited 5 minutes or so before clearing, that should be enough.
> If
> > > it's been 5 minutes since the last activity the connection should be
> > dead,
> > > unless you're shipping big files.
> > >
> > > Or, alternatively, in READ_REQUEST_HDR_HOOK pull the data out and
> attach
> > it
> > > to the txn, cleaning it up in TXN_CLOSE. Set a time stamp on entry in
> the
> > > table and if that's been 5 minutes, the connection has definitely timed
> > > out.
> > >
> >
>
Re: ssl conn and http
Posted by Alan Carroll <so...@oath.com.INVALID>.
I don't think I'd do it for TLS_CLOSE, but VCONN_CLOSE would likely be
worth it. There's an issue on that, linked from the issue I mentioned
above. It's about the same thing - plugin wants to attach data during TLS
handshake for later retrieval at the HTTP level that can be cleaned up if
something goes wrong.
On Wed, Sep 27, 2017 at 4:29 PM, Dk Jack <dn...@gmail.com> wrote:
> Yeah, I was thinking along similar lines...
>
> Do you think, it'd be a worth while effort to add a ssl-close event hook?
>
> On Wed, Sep 27, 2017 at 1:31 PM, Alan Carroll <
> solidwallofcode@oath.com.invalid> wrote:
>
> > I'd probably put a time stamp in and update it during the hook
> processing.
> > If you waited 5 minutes or so before clearing, that should be enough. If
> > it's been 5 minutes since the last activity the connection should be
> dead,
> > unless you're shipping big files.
> >
> > Or, alternatively, in READ_REQUEST_HDR_HOOK pull the data out and attach
> it
> > to the txn, cleaning it up in TXN_CLOSE. Set a time stamp on entry in the
> > table and if that's been 5 minutes, the connection has definitely timed
> > out.
> >
>
Re: ssl conn and http
Posted by Dk Jack <dn...@gmail.com>.
Yeah, I was thinking along similar lines...
Do you think, it'd be a worth while effort to add a ssl-close event hook?
On Wed, Sep 27, 2017 at 1:31 PM, Alan Carroll <
solidwallofcode@oath.com.invalid> wrote:
> I'd probably put a time stamp in and update it during the hook processing.
> If you waited 5 minutes or so before clearing, that should be enough. If
> it's been 5 minutes since the last activity the connection should be dead,
> unless you're shipping big files.
>
> Or, alternatively, in READ_REQUEST_HDR_HOOK pull the data out and attach it
> to the txn, cleaning it up in TXN_CLOSE. Set a time stamp on entry in the
> table and if that's been 5 minutes, the connection has definitely timed
> out.
>
Re: ssl conn and http
Posted by Alan Carroll <so...@oath.com.INVALID>.
I'd probably put a time stamp in and update it during the hook processing.
If you waited 5 minutes or so before clearing, that should be enough. If
it's been 5 minutes since the last activity the connection should be dead,
unless you're shipping big files.
Or, alternatively, in READ_REQUEST_HDR_HOOK pull the data out and attach it
to the txn, cleaning it up in TXN_CLOSE. Set a time stamp on entry in the
table and if that's been 5 minutes, the connection has definitely timed out.
Re: ssl conn and http
Posted by Dk Jack <dn...@gmail.com>.
I did try to see if there was a n openssl hook that gets invoked when the
connection
is closed. However, I couldn't find one. I guess it doesn't make sense to
have one from
openssl perspective since connection close is is always initiated by the
application.
What I need is a i.e a callback from SSLNetVConnection::do_io_close.
Otherwise, I'll
have to go with what you suggested i.e. maintain a table in the plugin. The
problem
I see with that is, when to expire (i.e. free) the entries in the table?
On Wed, Sep 27, 2017 at 4:42 AM, Alan Carroll <
solidwallofcode@oath.com.invalid> wrote:
> Unfortunately I don't know of a reliable way to do clean up, which is one
> reason I would think of a plugin maintained table which at least has the
> possibility of cleanup. This does point out a problem with the mentioned
> issue - if that is done there must be some reliable way to clean it up.
>
> I'm not an SSL expert but something dangerous you could try is to see if
> there is an openSSL hook you can grab to do the cleanup. The dangerous part
> is it is likely the ATS core already has a callback in place so you'd need
> to retrieve that and call it from your hook.
>
Re: ssl conn and http
Posted by Alan Carroll <so...@oath.com.INVALID>.
Unfortunately I don't know of a reliable way to do clean up, which is one
reason I would think of a plugin maintained table which at least has the
possibility of cleanup. This does point out a problem with the mentioned
issue - if that is done there must be some reliable way to clean it up.
I'm not an SSL expert but something dangerous you could try is to see if
there is an openSSL hook you can grab to do the cleanup. The dangerous part
is it is likely the ATS core already has a callback in place so you'd need
to retrieve that and call it from your hook.
Re: ssl conn and http
Posted by Dk Jack <dn...@gmail.com>.
Hi Alan,
Thanks for the response. After looking at the code a bit more, I figured I
can do something like this:
TSHttpHookAdd(TS_EVENT_VCONN_PRE_ACCEPT, pre_accept_cb);
pre_accept_callback(TSCont /*contp*/, TSEvent /*event*/, void *edata)
{
TSVConn ssl_vc = reinterpret_cast<TSVConn>(edata);
Connection sslobj = TSVConnSSLConnectionGet(ssl_vc);
TSSslSSL *ssl = reinterpret_cast<SSL *>(sslobj);
...
}
I can allocate some memory off the heap and then store my ssl contextual
stuff in the openssl
ssl object using ssl apis...
https://www.openssl.org/docs/manmaster/man3/SSL_set_ex_data.html
Later in my HTTP callback hooks, it looks like I can do the following to
retrieve the ssl object.
TSHttpTxn txn = static_cast<TSHttpTxn>(cppapi_txn.getAtsHandle()); //
TSHttpSsn ssn = TSHttpTxnSsnGet(txn);
SSL* ssl = (SSL*) TSHttpSsnSSLConnectionGet(ssn);
Once I have the ssl object, I can retrieve my ssl contextual data using the
ssl APIs.
Do you see any issues with this approach? One question I have is, is the
SSL object created
when the PRE_ACCEPT hook is called?
Another problem I see is, how do I free the memory I allocated to store
data in the ssl-object.
I'd like to free that when the SSL connection closes (for any reason:
normal close, error etc).
I don't see an event hook for SSL connection close. I
see TS_EVENT_HTTP_TXN_CLOSE
but I don't think that's equivalent to ssl connection close. Any
suggestions? Thanks.
Dk.
On Tue, Sep 26, 2017 at 6:54 AM, Alan Carroll <
solidwallofcode@oath.com.invalid> wrote:
> This is a known problem, see
> https://github.com/apache/trafficserver/issues/2388 . The point of that
> feature request is to be able to store data on the TSNetVConn (which you
> have in the SSL hook) and then retrieve it later in an HTTP hook (from
> which you can get to the same TSNetVConn). Currently I can't suggest
> anything better than making your own global table of TSNetVConn -> SSL
> information which admittedly is painful.
>
> On Mon, Sep 25, 2017 at 9:31 PM, Dk Jack <dn...@gmail.com> wrote:
>
> > Hi,
> > I am adding a hook to get access to the ssl handshake data. I am using
> > something like:
> >
> > TSHttpHookAdd(TS_SSL_CERT_HOOK, cb_cert);
> >
> > In the ssl callback, I am extracting some info from the ssl connection
> and
> > I'd like to save the info. I'd like to later retrieve this info when my
> > Http hook call back is invoked.
> >
> > TSHttpHookAdd(TS_HTTP_SEND_REQUEST_HDR_HOOK, cont);
> >
> > When my HTTP hook callback is called, I get TSHttpTxn. From HttpTxn how
> to
> > retrieve the saved ssl information? What is the correct way to make this
> > work? Any help is appreciated. Thanks.
> >
> > Dk.
> >
>
Re: ssl conn and http
Posted by Alan Carroll <so...@oath.com.INVALID>.
This is a known problem, see
https://github.com/apache/trafficserver/issues/2388 . The point of that
feature request is to be able to store data on the TSNetVConn (which you
have in the SSL hook) and then retrieve it later in an HTTP hook (from
which you can get to the same TSNetVConn). Currently I can't suggest
anything better than making your own global table of TSNetVConn -> SSL
information which admittedly is painful.
On Mon, Sep 25, 2017 at 9:31 PM, Dk Jack <dn...@gmail.com> wrote:
> Hi,
> I am adding a hook to get access to the ssl handshake data. I am using
> something like:
>
> TSHttpHookAdd(TS_SSL_CERT_HOOK, cb_cert);
>
> In the ssl callback, I am extracting some info from the ssl connection and
> I'd like to save the info. I'd like to later retrieve this info when my
> Http hook call back is invoked.
>
> TSHttpHookAdd(TS_HTTP_SEND_REQUEST_HDR_HOOK, cont);
>
> When my HTTP hook callback is called, I get TSHttpTxn. From HttpTxn how to
> retrieve the saved ssl information? What is the correct way to make this
> work? Any help is appreciated. Thanks.
>
> Dk.
>