You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@knox.apache.org by Rajesh Chandramohan <ra...@yahoo.com> on 2016/12/14 18:12:21 UTC
In Knox can we capture Bytes transfered info in logs
Hi Knox-team,
In knox can we capture the bytes transferred information along with file name etc. This would help us to track how much transaction each users are doing via knox , Any method or thoughts on this?
We make use of audit logs to capture each user how many transaction doing per hr per day. But we like to capture the details of how much bytes per transaction users are doing which will help us for various cases to manage Hadoop and Knox better. Any thoughts on this ? Thanks\Rajesh
Re: In Knox can we capture Bytes transfered info in logs
Posted by Rajesh Chandramohan <ra...@yahoo.com.INVALID>.
Hi ,
Opened a Knox jira [KNOX-822] In Knox can we capture Bytes transfered info in logs - ASF JIRA to track . Will look into the option
|
|
|
| | |
|
|
|
| |
[KNOX-822] In Knox can we capture Bytes transfered info in logs - ASF JIRA
| |
|
|
Thanks\Rajesh
On Wednesday, 14 December 2016 8:15 PM, larry mccay <lm...@apache.org> wrote:
Hello Rajesh -
I don't think that we have any way to do this today.There may be extension points in our metrics or in our audit logging to capture payload sizes.I imagine that the interesting place to do this would be in the dispatch classes to capture what is being sent to and received from the backend services.
This would require some investigation to determine whether we can gather the sizes and how best to capture them.
I would suggest filing an improvement JIRA and continuing the discussion there.
Thanks for your interest in Knox!
--larry
On Wed, Dec 14, 2016 at 1:12 PM, Rajesh Chandramohan <ra...@yahoo.com> wrote:
Hi Knox-team,
In knox can we capture the bytes transferred information along with file name etc. This would help us to track how much transaction each users are doing via knox , Any method or thoughts on this?
We make use of audit logs to capture each user how many transaction doing per hr per day. But we like to capture the details of how much bytes per transaction users are doing which will help us for various cases to manage Hadoop and Knox better. Any thoughts on this ? Thanks\Rajesh
Re: In Knox can we capture Bytes transfered info in logs
Posted by Rajesh Chandramohan <ra...@yahoo.com.INVALID>.
Thanks kevin/larry , for somepointer , will look into this ..for tracking purpose we opened a jira [KNOX-822] In Knox can we capture Bytes transfered info in logs - ASF JIRA
Lets explore the possibility and implementation .
|
|
|
| | |
|
|
|
| |
[KNOX-822] In Knox can we capture Bytes transfered info in logs - ASF JIRA
| |
|
|
Thanks\Rajesh
On Thursday, 15 December 2016 6:35 AM, Kevin Minder <ke...@oracle.com> wrote:
Actually if you enable the properties shown in gateway-log4j.properties and restart you should get request/response sizes in a separate access log but unfortunately not the authenticated/mapped user. Keep in mind that the request length might be based on the request Content-Length header. The code involved is in gateway-server/src/main/java/org/apache/hadoop/gateway/trace and AccessHandler.java in particular.
#log4j.logger.org.apache.hadoop.gateway.access=TRACE,httpaccess#log4j.additivity.org.apache.hadoop.gateway.access=false
#log4j.appender.httpaccess=org.apache.log4j.DailyRollingFileAppender#log4j.appender.httpaccess.File=${app.log.dir}/${launcher.name}-http-access.log#log4j.appender.httpaccess.DatePattern=.yyyy-MM-dd#log4j.appender.httpaccess.layout=org.apache.log4j.PatternLayout#log4j.appender.httpaccess.layout.ConversionPattern=%d{ISO8601}|%t|%m%n
On Dec 14, 2016, at 11:15 PM, larry mccay <lm...@apache.org> wrote:
Hello Rajesh -
I don't think that we have any way to do this today.There may be extension points in our metrics or in our audit logging to capture payload sizes.I imagine that the interesting place to do this would be in the dispatch classes to capture what is being sent to and received from the backend services.
This would require some investigation to determine whether we can gather the sizes and how best to capture them.
I would suggest filing an improvement JIRA and continuing the discussion there.
Thanks for your interest in Knox!
--larry
On Wed, Dec 14, 2016 at 1:12 PM, Rajesh Chandramohan <ra...@yahoo.com> wrote:
Hi Knox-team,
In knox can we capture the bytes transferred information along with file name etc. This would help us to track how much transaction each users are doing via knox , Any method or thoughts on this?
We make use of audit logs to capture each user how many transaction doing per hr per day. But we like to capture the details of how much bytes per transaction users are doing which will help us for various cases to manage Hadoop and Knox better. Any thoughts on this ? Thanks\Rajesh
Re: In Knox can we capture Bytes transfered info in logs
Posted by Rajesh Chandramohan <ra...@yahoo.com>.
Thanks kevin/larry , for somepointer , will look into this ..for tracking purpose we opened a jira [KNOX-822] In Knox can we capture Bytes transfered info in logs - ASF JIRA
Lets explore the possibility and implementation .
|
|
|
| | |
|
|
|
| |
[KNOX-822] In Knox can we capture Bytes transfered info in logs - ASF JIRA
| |
|
|
Thanks\Rajesh
On Thursday, 15 December 2016 6:35 AM, Kevin Minder <ke...@oracle.com> wrote:
Actually if you enable the properties shown in gateway-log4j.properties and restart you should get request/response sizes in a separate access log but unfortunately not the authenticated/mapped user. Keep in mind that the request length might be based on the request Content-Length header. The code involved is in gateway-server/src/main/java/org/apache/hadoop/gateway/trace and AccessHandler.java in particular.
#log4j.logger.org.apache.hadoop.gateway.access=TRACE,httpaccess#log4j.additivity.org.apache.hadoop.gateway.access=false
#log4j.appender.httpaccess=org.apache.log4j.DailyRollingFileAppender#log4j.appender.httpaccess.File=${app.log.dir}/${launcher.name}-http-access.log#log4j.appender.httpaccess.DatePattern=.yyyy-MM-dd#log4j.appender.httpaccess.layout=org.apache.log4j.PatternLayout#log4j.appender.httpaccess.layout.ConversionPattern=%d{ISO8601}|%t|%m%n
On Dec 14, 2016, at 11:15 PM, larry mccay <lm...@apache.org> wrote:
Hello Rajesh -
I don't think that we have any way to do this today.There may be extension points in our metrics or in our audit logging to capture payload sizes.I imagine that the interesting place to do this would be in the dispatch classes to capture what is being sent to and received from the backend services.
This would require some investigation to determine whether we can gather the sizes and how best to capture them.
I would suggest filing an improvement JIRA and continuing the discussion there.
Thanks for your interest in Knox!
--larry
On Wed, Dec 14, 2016 at 1:12 PM, Rajesh Chandramohan <ra...@yahoo.com> wrote:
Hi Knox-team,
In knox can we capture the bytes transferred information along with file name etc. This would help us to track how much transaction each users are doing via knox , Any method or thoughts on this?
We make use of audit logs to capture each user how many transaction doing per hr per day. But we like to capture the details of how much bytes per transaction users are doing which will help us for various cases to manage Hadoop and Knox better. Any thoughts on this ? Thanks\Rajesh
Re: In Knox can we capture Bytes transfered info in logs
Posted by Kevin Minder <ke...@oracle.com>.
Actually if you enable the properties shown in gateway-log4j.properties and restart you should get request/response sizes in a separate access log but unfortunately not the authenticated/mapped user. Keep in mind that the request length might be based on the request Content-Length header. The code involved is in gateway-server/src/main/java/org/apache/hadoop/gateway/trace and AccessHandler.java in particular.
#log4j.logger.org.apache.hadoop.gateway.access=TRACE,httpaccess
#log4j.additivity.org.apache.hadoop.gateway.access=false
#log4j.appender.httpaccess=org.apache.log4j.DailyRollingFileAppender
#log4j.appender.httpaccess.File=${app.log.dir}/${launcher.name}-http-access.log
#log4j.appender.httpaccess.DatePattern=.yyyy-MM-dd
#log4j.appender.httpaccess.layout=org.apache.log4j.PatternLayout
#log4j.appender.httpaccess.layout.ConversionPattern=%d{ISO8601}|%t|%m%n
> On Dec 14, 2016, at 11:15 PM, larry mccay <lmccay@apache.org <ma...@apache.org>> wrote:
>
> Hello Rajesh -
>
> I don't think that we have any way to do this today.
> There may be extension points in our metrics or in our audit logging to capture payload sizes.
> I imagine that the interesting place to do this would be in the dispatch classes to capture what is being sent to and received from the backend services.
>
> This would require some investigation to determine whether we can gather the sizes and how best to capture them.
>
> I would suggest filing an improvement JIRA and continuing the discussion there.
>
> Thanks for your interest in Knox!
>
> --larry
>
> On Wed, Dec 14, 2016 at 1:12 PM, Rajesh Chandramohan <rajesh_ymail@yahoo.com <ma...@yahoo.com>> wrote:
> Hi Knox-team,
>
> In knox can we capture the bytes transferred information along with file name etc. This would help us to track how much transaction each users are doing via knox , Any method or thoughts on this?
>
> We make use of audit logs to capture each user how many transaction doing per hr per day. But we like to capture the details of how much bytes per transaction users are doing which will help us for various cases to manage Hadoop and Knox better. Any thoughts on this ?
>
>
> Thanks
> \Rajesh
>
>
Re: In Knox can we capture Bytes transfered info in logs
Posted by Kevin Minder <ke...@oracle.com>.
Actually if you enable the properties shown in gateway-log4j.properties and restart you should get request/response sizes in a separate access log but unfortunately not the authenticated/mapped user. Keep in mind that the request length might be based on the request Content-Length header. The code involved is in gateway-server/src/main/java/org/apache/hadoop/gateway/trace and AccessHandler.java in particular.
#log4j.logger.org.apache.hadoop.gateway.access=TRACE,httpaccess
#log4j.additivity.org.apache.hadoop.gateway.access=false
#log4j.appender.httpaccess=org.apache.log4j.DailyRollingFileAppender
#log4j.appender.httpaccess.File=${app.log.dir}/${launcher.name}-http-access.log
#log4j.appender.httpaccess.DatePattern=.yyyy-MM-dd
#log4j.appender.httpaccess.layout=org.apache.log4j.PatternLayout
#log4j.appender.httpaccess.layout.ConversionPattern=%d{ISO8601}|%t|%m%n
> On Dec 14, 2016, at 11:15 PM, larry mccay <lmccay@apache.org <ma...@apache.org>> wrote:
>
> Hello Rajesh -
>
> I don't think that we have any way to do this today.
> There may be extension points in our metrics or in our audit logging to capture payload sizes.
> I imagine that the interesting place to do this would be in the dispatch classes to capture what is being sent to and received from the backend services.
>
> This would require some investigation to determine whether we can gather the sizes and how best to capture them.
>
> I would suggest filing an improvement JIRA and continuing the discussion there.
>
> Thanks for your interest in Knox!
>
> --larry
>
> On Wed, Dec 14, 2016 at 1:12 PM, Rajesh Chandramohan <rajesh_ymail@yahoo.com <ma...@yahoo.com>> wrote:
> Hi Knox-team,
>
> In knox can we capture the bytes transferred information along with file name etc. This would help us to track how much transaction each users are doing via knox , Any method or thoughts on this?
>
> We make use of audit logs to capture each user how many transaction doing per hr per day. But we like to capture the details of how much bytes per transaction users are doing which will help us for various cases to manage Hadoop and Knox better. Any thoughts on this ?
>
>
> Thanks
> \Rajesh
>
>
Re: In Knox can we capture Bytes transfered info in logs
Posted by Rajesh Chandramohan <ra...@yahoo.com>.
Hi ,
Opened a Knox jira [KNOX-822] In Knox can we capture Bytes transfered info in logs - ASF JIRA to track . Will look into the option
|
|
|
| | |
|
|
|
| |
[KNOX-822] In Knox can we capture Bytes transfered info in logs - ASF JIRA
| |
|
|
Thanks\Rajesh
On Wednesday, 14 December 2016 8:15 PM, larry mccay <lm...@apache.org> wrote:
Hello Rajesh -
I don't think that we have any way to do this today.There may be extension points in our metrics or in our audit logging to capture payload sizes.I imagine that the interesting place to do this would be in the dispatch classes to capture what is being sent to and received from the backend services.
This would require some investigation to determine whether we can gather the sizes and how best to capture them.
I would suggest filing an improvement JIRA and continuing the discussion there.
Thanks for your interest in Knox!
--larry
On Wed, Dec 14, 2016 at 1:12 PM, Rajesh Chandramohan <ra...@yahoo.com> wrote:
Hi Knox-team,
In knox can we capture the bytes transferred information along with file name etc. This would help us to track how much transaction each users are doing via knox , Any method or thoughts on this?
We make use of audit logs to capture each user how many transaction doing per hr per day. But we like to capture the details of how much bytes per transaction users are doing which will help us for various cases to manage Hadoop and Knox better. Any thoughts on this ? Thanks\Rajesh
Re: In Knox can we capture Bytes transfered info in logs
Posted by larry mccay <lm...@apache.org>.
Hello Rajesh -
I don't think that we have any way to do this today.
There may be extension points in our metrics or in our audit logging to
capture payload sizes.
I imagine that the interesting place to do this would be in the dispatch
classes to capture what is being sent to and received from the backend
services.
This would require some investigation to determine whether we can gather
the sizes and how best to capture them.
I would suggest filing an improvement JIRA and continuing the discussion
there.
Thanks for your interest in Knox!
--larry
On Wed, Dec 14, 2016 at 1:12 PM, Rajesh Chandramohan <rajesh_ymail@yahoo.com
> wrote:
> Hi Knox-team,
>
> In knox can we capture the bytes transferred information along with file
> name etc. This would help us to track how much transaction each users are
> doing via knox , Any method or thoughts on this?
>
> We make use of audit logs to capture each user how many transaction doing
> per hr per day. But we like to capture the details of how much bytes per
> transaction users are doing which will help us for various cases to manage
> Hadoop and Knox better. Any thoughts on this ?
>
>
> Thanks
> \Rajesh
>
>
Re: In Knox can we capture Bytes transfered info in logs
Posted by larry mccay <lm...@apache.org>.
Hello Rajesh -
I don't think that we have any way to do this today.
There may be extension points in our metrics or in our audit logging to
capture payload sizes.
I imagine that the interesting place to do this would be in the dispatch
classes to capture what is being sent to and received from the backend
services.
This would require some investigation to determine whether we can gather
the sizes and how best to capture them.
I would suggest filing an improvement JIRA and continuing the discussion
there.
Thanks for your interest in Knox!
--larry
On Wed, Dec 14, 2016 at 1:12 PM, Rajesh Chandramohan <rajesh_ymail@yahoo.com
> wrote:
> Hi Knox-team,
>
> In knox can we capture the bytes transferred information along with file
> name etc. This would help us to track how much transaction each users are
> doing via knox , Any method or thoughts on this?
>
> We make use of audit logs to capture each user how many transaction doing
> per hr per day. But we like to capture the details of how much bytes per
> transaction users are doing which will help us for various cases to manage
> Hadoop and Knox better. Any thoughts on this ?
>
>
> Thanks
> \Rajesh
>
>