You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Isaac Cruz <is...@dzsi.com> on 2022/09/23 10:31:36 UTC
HttpCore 5.1.4: Get stats of sent/rcvd bytes
Hi,
I want to get statistics of sent and received bytes from an HTTP server. I had a working implementation on HttpCore 4.x using NHttpServerConnection.getMetrics() that were added up on closing connection. Then I migrated to HttpCore 5.1.x and that API disappeared, so what I am using is AsyncServerBootstrap.setIOSessionDecorator() to set a custom class that will count bytes on read() and write() methods. This works perfectly fine when using plain connections, but when using HTTPS, read bytes won't work. When receiving an HTTP request (just before consumeRequest() is called), read() method is called, but returns 0 bytes. But then there are no more calls, even though I receive the calls to consume() with the received packets.
What I would expect is that the IOSession decorated would be the lowest layer, on encrypted socket, but it looks like it decorates a higher level IOSession, on inPlain (so it does not count handshake data), and additionally it is not being called for received data.
Is there any other way of getting those metrics?
Thanks and regards,
Isaac
Disclaimer
The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.
This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast, a leader in email security and cyber resilience. Mimecast integrates email defenses with brand protection, security awareness training, web security, compliance and other essential capabilities. Mimecast helps protect large and small organizations from malicious activity, human error and technology failure; and to lead the movement toward building a more resilient world. To find out more, visit our website.
RE: HttpCore 5.1.4: Get stats of sent/rcvd bytes
Posted by Isaac Cruz <is...@dzsi.com>.
Main limitation is that I can't find a way to obtain that EndpointDetails object when connection closes, using IOSessionListener.
-----Original Message-----
From: Oleg Kalnichevski <ol...@apache.org>
Sent: domingo, 25 de septiembre de 2022 16:39
To: HttpComponents Project <de...@hc.apache.org>
Subject: Re: HttpCore 5.1.4: Get stats of sent/rcvd bytes
On Sun, 2022-09-25 at 05:27 +0000, Isaac Cruz wrote:
> Sorry, I did not explain well, let me elaborate.
>
> On 4.x I was extending HttpAsyncService to override closed() method,
> and as the parameter was a NHttpServerConnection object I could obtain
> metrics from it.
>
> On 5.x the most similar method I found was setIOSessionListener, but
> the disconnected callback receive an IOSession parameter, and there is
> no metrics there. So I have not found yet an easy way of getting
> metrics on this version.
>
> Thanks,
> Isaac
>
I am sorry I am not sure I understand the problem. I am not sure what stops you from reading sent / received bytes data from EndpointDetails
Oleg.
> -----Original Message-----
> From: Oleg Kalnichevski <ol...@apache.org>
> Sent: sábado, 24 de septiembre de 2022 11:00
> To: HttpComponents Project <de...@hc.apache.org>
> Subject: Re: HttpCore 5.1.4: Get stats of sent/rcvd bytes
>
> On Fri, 2022-09-23 at 10:31 +0000, Isaac Cruz wrote:
> > Hi,
> >
> > I want to get statistics of sent and received bytes from an HTTP
> > server. I had a working implementation on HttpCore 4.x using
> > NHttpServerConnection.getMetrics() that were added up on closing
> > connection. Then I migrated to HttpCore 5.1.x and that API
> > disappeared,
>
> Why disappeared? Connection and transport metrics are still accessible
> through the execution context. What is wrong with using
> HttpConnectionMetrics or EndpointDetails?
>
>
> > so what I am using is AsyncServerBootstrap.setIOSessionDecorator()
> > to
> > set a custom class that will count bytes on read() and write()
> > methods. This works perfectly fine when using plain connections, but
> > when using HTTPS, read bytes won't work. When receiving an HTTP
> > request (just before consumeRequest() is called), read() method is
> > called, but returns 0 bytes. But then there are no more calls, even
> > though I receive the calls to consume() with the received packets.
> >
> > What I would expect is that the IOSession decorated would be the
> > lowest layer, on encrypted socket, but it looks like it decorates a
> > higher level IOSession, on inPlain (so it does not count handshake
> > data), and additionally it is not being called for received data.
> >
> > Is there any other way of getting those metrics?
> >
>
> IOSessionDecorator was initially intended for injecting i/o session
> level logging at in order to avoid direct dependency on any logging
> framework. What one usually wants to see in the logs is plain data in
> a human readable form and not raw TLS data. For that reason the i/o
> session events are intercepted at the entry to the protocol layer
> post- TLS, and not pre-TLS. I do not think we should change that.
>
> I hope this helps.
>
> If still want to introduce an interception point at the i/o channel
> level, we can discuss it but do hope it is not going to be necessary.
>
> Cheers
>
> Oleg
>
> > Thanks and regards,
> > Isaac
> >
> > Disclaimer
> >
> > The information contained in this communication from the sender is
> > confidential. It is intended solely for use by the recipient and
> > others authorized to receive it. If you are not the recipient, you
> > are hereby notified that any disclosure, copying, distribution or
> > taking action in relation of the contents of this information is
> > strictly prohibited and may be unlawful.
> >
> > This email has been scanned for viruses and malware, and may have
> > been automatically archived by Mimecast, a leader in email security
> > and cyber resilience. Mimecast integrates email defenses with brand
> > protection, security awareness training, web security, compliance
> > and other essential capabilities. Mimecast helps protect large and
> > small organizations from malicious activity, human error and
> > technology failure; and to lead the movement toward building a more
> > resilient world. To find out more, visit our website.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org For additional
> commands, e-mail: dev-help@hc.apache.org
>
> Disclaimer
>
> The information contained in this communication from the sender is
> confidential. It is intended solely for use by the recipient and
> others authorized to receive it. If you are not the recipient, you are
> hereby notified that any disclosure, copying, distribution or taking
> action in relation of the contents of this information is strictly
> prohibited and may be unlawful.
>
> This email has been scanned for viruses and malware, and may have been
> automatically archived by Mimecast, a leader in email security and
> cyber resilience. Mimecast integrates email defenses with brand
> protection, security awareness training, web security, compliance and
> other essential capabilities. Mimecast helps protect large and small
> organizations from malicious activity, human error and technology
> failure; and to lead the movement toward building a more resilient
> world. To find out more, visit our website.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org For additional commands, e-mail: dev-help@hc.apache.org
Disclaimer
The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.
This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast, a leader in email security and cyber resilience. Mimecast integrates email defenses with brand protection, security awareness training, web security, compliance and other essential capabilities. Mimecast helps protect large and small organizations from malicious activity, human error and technology failure; and to lead the movement toward building a more resilient world. To find out more, visit our website.
Re: HttpCore 5.1.4: Get stats of sent/rcvd bytes
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, 2022-09-25 at 05:27 +0000, Isaac Cruz wrote:
> Sorry, I did not explain well, let me elaborate.
>
> On 4.x I was extending HttpAsyncService to override closed() method,
> and as the parameter was a NHttpServerConnection object I could
> obtain metrics from it.
>
> On 5.x the most similar method I found was setIOSessionListener, but
> the disconnected callback receive an IOSession parameter, and there
> is no metrics there. So I have not found yet an easy way of getting
> metrics on this version.
>
> Thanks,
> Isaac
>
I am sorry I am not sure I understand the problem. I am not sure what
stops you from reading sent / received bytes data from EndpointDetails
Oleg.
> -----Original Message-----
> From: Oleg Kalnichevski <ol...@apache.org>
> Sent: sábado, 24 de septiembre de 2022 11:00
> To: HttpComponents Project <de...@hc.apache.org>
> Subject: Re: HttpCore 5.1.4: Get stats of sent/rcvd bytes
>
> On Fri, 2022-09-23 at 10:31 +0000, Isaac Cruz wrote:
> > Hi,
> >
> > I want to get statistics of sent and received bytes from an HTTP
> > server. I had a working implementation on HttpCore 4.x using
> > NHttpServerConnection.getMetrics() that were added up on closing
> > connection. Then I migrated to HttpCore 5.1.x and that API
> > disappeared,
>
> Why disappeared? Connection and transport metrics are still
> accessible through the execution context. What is wrong with using
> HttpConnectionMetrics or EndpointDetails?
>
>
> > so what I am using is AsyncServerBootstrap.setIOSessionDecorator()
> > to
> > set a custom class that will count bytes on read() and write()
> > methods. This works perfectly fine when using plain connections,
> > but
> > when using HTTPS, read bytes won't work. When receiving an HTTP
> > request (just before consumeRequest() is called), read() method is
> > called, but returns 0 bytes. But then there are no more calls, even
> > though I receive the calls to consume() with the received packets.
> >
> > What I would expect is that the IOSession decorated would be the
> > lowest layer, on encrypted socket, but it looks like it decorates a
> > higher level IOSession, on inPlain (so it does not count handshake
> > data), and additionally it is not being called for received data.
> >
> > Is there any other way of getting those metrics?
> >
>
> IOSessionDecorator was initially intended for injecting i/o session
> level logging at in order to avoid direct dependency on any logging
> framework. What one usually wants to see in the logs is plain data in
> a human readable form and not raw TLS data. For that reason the i/o
> session events are intercepted at the entry to the protocol layer
> post- TLS, and not pre-TLS. I do not think we should change that.
>
> I hope this helps.
>
> If still want to introduce an interception point at the i/o channel
> level, we can discuss it but do hope it is not going to be necessary.
>
> Cheers
>
> Oleg
>
> > Thanks and regards,
> > Isaac
> >
> > Disclaimer
> >
> > The information contained in this communication from the sender is
> > confidential. It is intended solely for use by the recipient and
> > others authorized to receive it. If you are not the recipient, you
> > are
> > hereby notified that any disclosure, copying, distribution or
> > taking
> > action in relation of the contents of this information is strictly
> > prohibited and may be unlawful.
> >
> > This email has been scanned for viruses and malware, and may have
> > been
> > automatically archived by Mimecast, a leader in email security and
> > cyber resilience. Mimecast integrates email defenses with brand
> > protection, security awareness training, web security, compliance
> > and
> > other essential capabilities. Mimecast helps protect large and
> > small
> > organizations from malicious activity, human error and technology
> > failure; and to lead the movement toward building a more resilient
> > world. To find out more, visit our website.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org For additional
> commands, e-mail: dev-help@hc.apache.org
>
> Disclaimer
>
> The information contained in this communication from the sender is
> confidential. It is intended solely for use by the recipient and
> others authorized to receive it. If you are not the recipient, you
> are hereby notified that any disclosure, copying, distribution or
> taking action in relation of the contents of this information is
> strictly prohibited and may be unlawful.
>
> This email has been scanned for viruses and malware, and may have
> been automatically archived by Mimecast, a leader in email security
> and cyber resilience. Mimecast integrates email defenses with brand
> protection, security awareness training, web security, compliance and
> other essential capabilities. Mimecast helps protect large and small
> organizations from malicious activity, human error and technology
> failure; and to lead the movement toward building a more resilient
> world. To find out more, visit our website.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
RE: HttpCore 5.1.4: Get stats of sent/rcvd bytes
Posted by Isaac Cruz <is...@dzsi.com>.
Sorry, I did not explain well, let me elaborate.
On 4.x I was extending HttpAsyncService to override closed() method, and as the parameter was a NHttpServerConnection object I could obtain metrics from it.
On 5.x the most similar method I found was setIOSessionListener, but the disconnected callback receive an IOSession parameter, and there is no metrics there. So I have not found yet an easy way of getting metrics on this version.
Thanks,
Isaac
-----Original Message-----
From: Oleg Kalnichevski <ol...@apache.org>
Sent: sábado, 24 de septiembre de 2022 11:00
To: HttpComponents Project <de...@hc.apache.org>
Subject: Re: HttpCore 5.1.4: Get stats of sent/rcvd bytes
On Fri, 2022-09-23 at 10:31 +0000, Isaac Cruz wrote:
> Hi,
>
> I want to get statistics of sent and received bytes from an HTTP
> server. I had a working implementation on HttpCore 4.x using
> NHttpServerConnection.getMetrics() that were added up on closing
> connection. Then I migrated to HttpCore 5.1.x and that API
> disappeared,
Why disappeared? Connection and transport metrics are still accessible through the execution context. What is wrong with using
HttpConnectionMetrics or EndpointDetails?
> so what I am using is AsyncServerBootstrap.setIOSessionDecorator() to
> set a custom class that will count bytes on read() and write()
> methods. This works perfectly fine when using plain connections, but
> when using HTTPS, read bytes won't work. When receiving an HTTP
> request (just before consumeRequest() is called), read() method is
> called, but returns 0 bytes. But then there are no more calls, even
> though I receive the calls to consume() with the received packets.
>
> What I would expect is that the IOSession decorated would be the
> lowest layer, on encrypted socket, but it looks like it decorates a
> higher level IOSession, on inPlain (so it does not count handshake
> data), and additionally it is not being called for received data.
>
> Is there any other way of getting those metrics?
>
IOSessionDecorator was initially intended for injecting i/o session level logging at in order to avoid direct dependency on any logging framework. What one usually wants to see in the logs is plain data in a human readable form and not raw TLS data. For that reason the i/o session events are intercepted at the entry to the protocol layer post- TLS, and not pre-TLS. I do not think we should change that.
I hope this helps.
If still want to introduce an interception point at the i/o channel level, we can discuss it but do hope it is not going to be necessary.
Cheers
Oleg
> Thanks and regards,
> Isaac
>
> Disclaimer
>
> The information contained in this communication from the sender is
> confidential. It is intended solely for use by the recipient and
> others authorized to receive it. If you are not the recipient, you are
> hereby notified that any disclosure, copying, distribution or taking
> action in relation of the contents of this information is strictly
> prohibited and may be unlawful.
>
> This email has been scanned for viruses and malware, and may have been
> automatically archived by Mimecast, a leader in email security and
> cyber resilience. Mimecast integrates email defenses with brand
> protection, security awareness training, web security, compliance and
> other essential capabilities. Mimecast helps protect large and small
> organizations from malicious activity, human error and technology
> failure; and to lead the movement toward building a more resilient
> world. To find out more, visit our website.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org For additional commands, e-mail: dev-help@hc.apache.org
Disclaimer
The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.
This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast, a leader in email security and cyber resilience. Mimecast integrates email defenses with brand protection, security awareness training, web security, compliance and other essential capabilities. Mimecast helps protect large and small organizations from malicious activity, human error and technology failure; and to lead the movement toward building a more resilient world. To find out more, visit our website.
Re: HttpCore 5.1.4: Get stats of sent/rcvd bytes
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Fri, 2022-09-23 at 10:31 +0000, Isaac Cruz wrote:
> Hi,
>
> I want to get statistics of sent and received bytes from an HTTP
> server. I had a working implementation on HttpCore 4.x using
> NHttpServerConnection.getMetrics() that were added up on closing
> connection. Then I migrated to HttpCore 5.1.x and that API
> disappeared,
Why disappeared? Connection and transport metrics are still accessible
through the execution context. What is wrong with using
HttpConnectionMetrics or EndpointDetails?
> so what I am using is AsyncServerBootstrap.setIOSessionDecorator() to
> set a custom class that will count bytes on read() and write()
> methods. This works perfectly fine when using plain connections, but
> when using HTTPS, read bytes won't work. When receiving an HTTP
> request (just before consumeRequest() is called), read() method is
> called, but returns 0 bytes. But then there are no more calls, even
> though I receive the calls to consume() with the received packets.
>
> What I would expect is that the IOSession decorated would be the
> lowest layer, on encrypted socket, but it looks like it decorates a
> higher level IOSession, on inPlain (so it does not count handshake
> data), and additionally it is not being called for received data.
>
> Is there any other way of getting those metrics?
>
IOSessionDecorator was initially intended for injecting i/o session
level logging at in order to avoid direct dependency on any logging
framework. What one usually wants to see in the logs is plain data in a
human readable form and not raw TLS data. For that reason the i/o
session events are intercepted at the entry to the protocol layer post-
TLS, and not pre-TLS. I do not think we should change that.
I hope this helps.
If still want to introduce an interception point at the i/o channel
level, we can discuss it but do hope it is not going to be necessary.
Cheers
Oleg
> Thanks and regards,
> Isaac
>
> Disclaimer
>
> The information contained in this communication from the sender is
> confidential. It is intended solely for use by the recipient and
> others authorized to receive it. If you are not the recipient, you
> are hereby notified that any disclosure, copying, distribution or
> taking action in relation of the contents of this information is
> strictly prohibited and may be unlawful.
>
> This email has been scanned for viruses and malware, and may have
> been automatically archived by Mimecast, a leader in email security
> and cyber resilience. Mimecast integrates email defenses with brand
> protection, security awareness training, web security, compliance and
> other essential capabilities. Mimecast helps protect large and small
> organizations from malicious activity, human error and technology
> failure; and to lead the movement toward building a more resilient
> world. To find out more, visit our website.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org