You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Paul Angus <pa...@shapeblue.com> on 2016/04/14 14:49:44 UTC
[discuss] CloudStack logging
Hi All
I think that we can all agree that CloudStack logs are very difficult to read, especially for operational staff.
I believe that the primary reason for this is that a large amount of what should be INFO level events are categorised as DEBUG. The logging therefore must be set at DEBUG for the logs to capture these events. By defaulting the logging to DEBUG we include a lot of detail which is counter-productive when using the logs to diagnose operational issues.
I think the situation can be greatly improved by reviewing the categorisation of logged events and where necessary re-categorise then such that INFO, WARN and ERROR logging would be sufficient for an 'operational' admin.
I think a phased approach where the default logging level remains at DEBUG while re-categorisation occurs would mean that nothing materially changes until we are happy with the categorisations. Then we can default the logging level to INFO.
Thoughts everyone?
Kind regards,
Paul Angus
Regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
Re: [discuss] CloudStack logging
Posted by Koushik Das <ko...@accelerite.com>.
Based on the namespace log level can be set. It needs to be configured in the log4j.
Check client/tomcatconf/log4j-cloud.xml.in in the source code and you will get an idea.
-Koushik
________________________________________
From: williamstevens@gmail.com <wi...@gmail.com> on behalf of Will Stevens <ws...@cloudops.com>
Sent: Thursday, April 14, 2016 7:29 PM
To: dev@cloudstack.apache.org
Subject: Re: [discuss] CloudStack logging
Do you have to recompile in order to turn off the logging for a specific
package or class? If yes, that is a show stopper for almost everyone...
If it only requires a management server restart, that is more realistic.
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Thu, Apr 14, 2016 at 9:48 AM, Daan Hoogland <da...@gmail.com>
wrote:
> On Thu, Apr 14, 2016 at 3:41 PM, Will Stevens <wi...@gmail.com>
> wrote:
>
> > Is there an easy way to do that Daan, or is it a tedious task you just
> have
> > to power through?
> >
> It is hard work, mostly tedious, sometimes hilarious and most definitely
>
> devops! It certainly wouldn't classify as development.
>
>
>
>
> --
> Daan
>
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
Re: [discuss] CloudStack logging
Posted by Will Stevens <ws...@cloudops.com>.
Do you have to recompile in order to turn off the logging for a specific
package or class? If yes, that is a show stopper for almost everyone...
If it only requires a management server restart, that is more realistic.
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Thu, Apr 14, 2016 at 9:48 AM, Daan Hoogland <da...@gmail.com>
wrote:
> On Thu, Apr 14, 2016 at 3:41 PM, Will Stevens <wi...@gmail.com>
> wrote:
>
> > Is there an easy way to do that Daan, or is it a tedious task you just
> have
> > to power through?
> >
> It is hard work, mostly tedious, sometimes hilarious and most definitely
>
> devops! It certainly wouldn't classify as development.
>
>
>
>
> --
> Daan
>
Re: [discuss] CloudStack logging
Posted by Daan Hoogland <da...@gmail.com>.
On Thu, Apr 14, 2016 at 3:41 PM, Will Stevens <wi...@gmail.com>
wrote:
> Is there an easy way to do that Daan, or is it a tedious task you just have
> to power through?
>
It is hard work, mostly tedious, sometimes hilarious and most definitely
devops! It certainly wouldn't classify as development.
--
Daan
Re: [discuss] CloudStack logging
Posted by Will Stevens <wi...@gmail.com>.
Is there an easy way to do that Daan, or is it a tedious task you just have
to power through?
I agree this initiative would be helpful.
On Apr 14, 2016 9:04 AM, "Daan Hoogland" <da...@gmail.com> wrote:
> I largely agree with just one sneer to the operators amongst us: the level
> can be set on a per package and even class basis. So you can work through
> disabling anoying logging step by step. Makes sense to keep looking at the
> source in the meanwhile to make sure sensible diagnostics remains
> available.
>
> On Thu, Apr 14, 2016 at 2:49 PM, Paul Angus <pa...@shapeblue.com>
> wrote:
>
> > Hi All
> >
> > I think that we can all agree that CloudStack logs are very difficult to
> > read, especially for operational staff.
> >
> > I believe that the primary reason for this is that a large amount of what
> > should be INFO level events are categorised as DEBUG. The logging
> > therefore must be set at DEBUG for the logs to capture these events. By
> > defaulting the logging to DEBUG we include a lot of detail which is
> > counter-productive when using the logs to diagnose operational issues.
> >
> > I think the situation can be greatly improved by reviewing the
> > categorisation of logged events and where necessary re-categorise then
> such
> > that INFO, WARN and ERROR logging would be sufficient for an
> 'operational'
> > admin.
> >
> > I think a phased approach where the default logging level remains at
> DEBUG
> > while re-categorisation occurs would mean that nothing materially changes
> > until we are happy with the categorisations. Then we can default the
> > logging level to INFO.
> >
> >
> >
> > Thoughts everyone?
> >
> >
> >
> >
> > Kind regards,
> >
> > Paul Angus
> >
> >
> > Regards,
> >
> > Paul Angus
> >
> > paul.angus@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> > @shapeblue
> >
>
>
>
> --
> Daan
>
Re: [discuss] CloudStack logging
Posted by Daan Hoogland <da...@gmail.com>.
I largely agree with just one sneer to the operators amongst us: the level
can be set on a per package and even class basis. So you can work through
disabling anoying logging step by step. Makes sense to keep looking at the
source in the meanwhile to make sure sensible diagnostics remains available.
On Thu, Apr 14, 2016 at 2:49 PM, Paul Angus <pa...@shapeblue.com>
wrote:
> Hi All
>
> I think that we can all agree that CloudStack logs are very difficult to
> read, especially for operational staff.
>
> I believe that the primary reason for this is that a large amount of what
> should be INFO level events are categorised as DEBUG. The logging
> therefore must be set at DEBUG for the logs to capture these events. By
> defaulting the logging to DEBUG we include a lot of detail which is
> counter-productive when using the logs to diagnose operational issues.
>
> I think the situation can be greatly improved by reviewing the
> categorisation of logged events and where necessary re-categorise then such
> that INFO, WARN and ERROR logging would be sufficient for an 'operational'
> admin.
>
> I think a phased approach where the default logging level remains at DEBUG
> while re-categorisation occurs would mean that nothing materially changes
> until we are happy with the categorisations. Then we can default the
> logging level to INFO.
>
>
>
> Thoughts everyone?
>
>
>
>
> Kind regards,
>
> Paul Angus
>
>
> Regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
--
Daan
Re: [discuss] CloudStack logging
Posted by Patrick Dube <pa...@gmail.com>.
Really agree with this one. With a toolchain like ELK, it isn't as bad
because you can filter out the things you don't want, but generally
speaking, the log levels need to be reevaluated. I would also add that many
log statements would benefit from additional context, but that is a whole
different story.
Another example:
INFO [com.cloud.agent.transport.Request] (StatsCollector-3:ctx-de473a88)
not building log message for '[{}]', _cmds.length == 1
It even says "not building log message".
On Thu, Apr 14, 2016 at 10:18 AM, Simon Weller <sw...@ena.com> wrote:
> I'm all for this as well.
>
> Training our systems folks on how to work through the logs is a pain.
> Focusing in on the most important items is more useful and also makes it
> easier to automate log fetching and categorization.
>
> - Si
>
> ________________________________________
> From: Paul Angus <pa...@shapeblue.com>
> Sent: Thursday, April 14, 2016 9:06 AM
> To: Wido den Hollander; dev@cloudstack.apache.org
> Subject: RE: [discuss] CloudStack logging
>
> Grabbing a couple of examples - these should be debug - I can't 'do'
> anything with this information.
>
> INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) Begin cleanup expired async-jobs
> INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) End cleanup expired async-jobs
> INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-2:ctx-3209f65c) Scan
> hung worker VM to recycle
> INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-113:ctx-56d9f9f2)
> Scan hung worker VM to recycle
>
> Whereas this should be WARN. I wouldn't want to lose this by switching off
> DEBUG
>
> DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-91226be7)
> Detected management node left, id:2, nodeIP:10.2.0.6
> DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-0bccd078)
> Detected management node left, id:2, nodeIP:10.2.0.6
>
>
>
> Kind regards,
>
> Paul Angus
>
> Regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
> -----Original Message-----
> From: Wido den Hollander [mailto:wido@widodh.nl]
> Sent: 14 April 2016 15:04
> To: Paul Angus <pa...@shapeblue.com>; dev@cloudstack.apache.org
> Subject: Re: [discuss] CloudStack logging
>
>
> > Op 14 april 2016 om 14:49 schreef Paul Angus <pa...@shapeblue.com>:
> >
> >
> > Hi All
> >
> > I think that we can all agree that CloudStack logs are very difficult
> > to read, especially for operational staff.
> >
> > I believe that the primary reason for this is that a large amount of
> > what should be INFO level events are categorised as DEBUG. The
> > logging therefore must be set at DEBUG for the logs to capture these
> > events. By defaulting the logging to DEBUG we include a lot of detail
> > which is counter-productive when using the logs to diagnose operational
> issues.
> >
>
> Yes!
>
> > I think the situation can be greatly improved by reviewing the
> > categorisation of logged events and where necessary re-categorise then
> > such that INFO, WARN and ERROR logging would be sufficient for an
> 'operational' admin.
> >
> > I think a phased approach where the default logging level remains at
> > DEBUG while re-categorisation occurs would mean that nothing
> > materially changes until we are happy with the categorisations. Then
> > we can default the logging level to INFO.
> >
>
> I created a issue for this a while ago:
> https://issues.apache.org/jira/browse/CLOUDSTACK-8645
>
> Short: Yes. Just change this where needed.
>
> Wido
>
> >
> >
> > Thoughts everyone?
> >
> >
> >
> >
> > Kind regards,
> >
> > Paul Angus
> >
> >
> > Regards,
> >
> > Paul Angus
> >
> > paul.angus@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
>
Re: [discuss] CloudStack logging
Posted by Simon Weller <sw...@ena.com>.
I'm all for this as well.
Training our systems folks on how to work through the logs is a pain. Focusing in on the most important items is more useful and also makes it easier to automate log fetching and categorization.
- Si
________________________________________
From: Paul Angus <pa...@shapeblue.com>
Sent: Thursday, April 14, 2016 9:06 AM
To: Wido den Hollander; dev@cloudstack.apache.org
Subject: RE: [discuss] CloudStack logging
Grabbing a couple of examples - these should be debug - I can't 'do' anything with this information.
INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) Begin cleanup expired async-jobs
INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) End cleanup expired async-jobs
INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-2:ctx-3209f65c) Scan hung worker VM to recycle
INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-113:ctx-56d9f9f2) Scan hung worker VM to recycle
Whereas this should be WARN. I wouldn't want to lose this by switching off DEBUG
DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-91226be7) Detected management node left, id:2, nodeIP:10.2.0.6
DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-0bccd078) Detected management node left, id:2, nodeIP:10.2.0.6
Kind regards,
Paul Angus
Regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
-----Original Message-----
From: Wido den Hollander [mailto:wido@widodh.nl]
Sent: 14 April 2016 15:04
To: Paul Angus <pa...@shapeblue.com>; dev@cloudstack.apache.org
Subject: Re: [discuss] CloudStack logging
> Op 14 april 2016 om 14:49 schreef Paul Angus <pa...@shapeblue.com>:
>
>
> Hi All
>
> I think that we can all agree that CloudStack logs are very difficult
> to read, especially for operational staff.
>
> I believe that the primary reason for this is that a large amount of
> what should be INFO level events are categorised as DEBUG. The
> logging therefore must be set at DEBUG for the logs to capture these
> events. By defaulting the logging to DEBUG we include a lot of detail
> which is counter-productive when using the logs to diagnose operational issues.
>
Yes!
> I think the situation can be greatly improved by reviewing the
> categorisation of logged events and where necessary re-categorise then
> such that INFO, WARN and ERROR logging would be sufficient for an 'operational' admin.
>
> I think a phased approach where the default logging level remains at
> DEBUG while re-categorisation occurs would mean that nothing
> materially changes until we are happy with the categorisations. Then
> we can default the logging level to INFO.
>
I created a issue for this a while ago:
https://issues.apache.org/jira/browse/CLOUDSTACK-8645
Short: Yes. Just change this where needed.
Wido
>
>
> Thoughts everyone?
>
>
>
>
> Kind regards,
>
> Paul Angus
>
>
> Regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
RE: [discuss] CloudStack logging
Posted by Paul Angus <pa...@shapeblue.com>.
Grabbing a couple of examples - these should be debug - I can't 'do' anything with this information.
INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) Begin cleanup expired async-jobs
INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) End cleanup expired async-jobs
INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-2:ctx-3209f65c) Scan hung worker VM to recycle
INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-113:ctx-56d9f9f2) Scan hung worker VM to recycle
Whereas this should be WARN. I wouldn't want to lose this by switching off DEBUG
DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-91226be7) Detected management node left, id:2, nodeIP:10.2.0.6
DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-0bccd078) Detected management node left, id:2, nodeIP:10.2.0.6
Kind regards,
Paul Angus
Regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
-----Original Message-----
From: Wido den Hollander [mailto:wido@widodh.nl]
Sent: 14 April 2016 15:04
To: Paul Angus <pa...@shapeblue.com>; dev@cloudstack.apache.org
Subject: Re: [discuss] CloudStack logging
> Op 14 april 2016 om 14:49 schreef Paul Angus <pa...@shapeblue.com>:
>
>
> Hi All
>
> I think that we can all agree that CloudStack logs are very difficult
> to read, especially for operational staff.
>
> I believe that the primary reason for this is that a large amount of
> what should be INFO level events are categorised as DEBUG. The
> logging therefore must be set at DEBUG for the logs to capture these
> events. By defaulting the logging to DEBUG we include a lot of detail
> which is counter-productive when using the logs to diagnose operational issues.
>
Yes!
> I think the situation can be greatly improved by reviewing the
> categorisation of logged events and where necessary re-categorise then
> such that INFO, WARN and ERROR logging would be sufficient for an 'operational' admin.
>
> I think a phased approach where the default logging level remains at
> DEBUG while re-categorisation occurs would mean that nothing
> materially changes until we are happy with the categorisations. Then
> we can default the logging level to INFO.
>
I created a issue for this a while ago:
https://issues.apache.org/jira/browse/CLOUDSTACK-8645
Short: Yes. Just change this where needed.
Wido
>
>
> Thoughts everyone?
>
>
>
>
> Kind regards,
>
> Paul Angus
>
>
> Regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
RE: [discuss] CloudStack logging
Posted by Paul Angus <pa...@shapeblue.com>.
Hi Pierre-Luc,
I have a pull request in to fix the rotating of the catalina.out logs - I'll check what it's status is.
And yes, being able to view VR, CPVM and SSVM logs from mgmt. server would be awesome. Some of our clients have 1000s of VRs so, some real thought needs to go into it...
Kind regards,
Paul Angus
Regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
-----Original Message-----
From: Pierre-Luc Dion [mailto:pdion@cloudops.com]
Sent: 15 April 2016 13:57
To: dev@cloudstack.apache.org
Subject: Re: [discuss] CloudStack logging
Something that could be interesting also is to remove VR and systemVMs logs from the cloudstack-management.log and have them into a separate files. I'm still not sure of those logs are traps only when there is problems, if it's the case it make sense to keep them in cloudstack-management.log.
Anyhow, it might be interesting to collect VR and systemVMs logs from the management servers.
The log structure is also pretty stable so far which make integration with ELK fairly easy which is good!
What about the catalina.out? is there any way to reduce is size?
I'll update the JIRA issue with problematic troubleshooting logs that are not clear to interpret...
Thanks Paul for taking that one !
PL
On Fri, Apr 15, 2016 at 4:44 AM, Daan Hoogland <da...@gmail.com>
wrote:
> Paul,
>
> I think that asking for input is a good approach. Those that have
> problem with a certain log message or - sequence are the best to say
> what should be different.
>
> Whether we should make hundreds of PRs or one?.. neither. I would say
> change anything from a big class to a small package, or get the flow
> for one API call and change anything in that. That would be cross
> package and may require classes in the management -, service -,
> orchestration and resource layers to be revisited for several flows.
>
>
> On Fri, Apr 15, 2016 at 10:06 AM, Paul Angus
> <pa...@shapeblue.com>
> wrote:
>
> > Great, I'll pick up the issue that Wido has already created
> > https://issues.apache.org/jira/browse/CLOUDSTACK-8645
> >
> > Does anyone have an suggestions wrt to structure of pull requests?
> > I'm inclinded to ask the community for copies of logs - particularly
> > ones that people have found difficult to read so that I or others
> > can work through them looking for anamolies - it's very messy from a
> > thoroughness point of view - but it would return the most useful
> > changes first. It
> also
> > puts the error msg in a context to make the categorisation more acurate.
> >
> >
> >
> > Kind regards,
> >
> > Paul Angus
> >
> > Regards,
> >
> > Paul Angus
> >
> > paul.angus@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
> >
> > -----Original Message-----
> > From: Wido den Hollander [mailto:wido@widodh.nl]
> > Sent: 14 April 2016 15:04
> > To: Paul Angus <pa...@shapeblue.com>; dev@cloudstack.apache.org
> > Subject: Re: [discuss] CloudStack logging
> >
> >
> > > Op 14 april 2016 om 14:49 schreef Paul Angus
> > > <paul.angus@shapeblue.com
> >:
> > >
> > >
> > > Hi All
> > >
> > > I think that we can all agree that CloudStack logs are very
> > > difficult to read, especially for operational staff.
> > >
> > > I believe that the primary reason for this is that a large amount
> > > of what should be INFO level events are categorised as DEBUG. The
> > > logging therefore must be set at DEBUG for the logs to capture
> > > these events. By defaulting the logging to DEBUG we include a lot
> > > of detail which is counter-productive when using the logs to
> > > diagnose operational
> > issues.
> > >
> >
> > Yes!
> >
> > > I think the situation can be greatly improved by reviewing the
> > > categorisation of logged events and where necessary re-categorise
> > > then such that INFO, WARN and ERROR logging would be sufficient
> > > for an
> > 'operational' admin.
> > >
> > > I think a phased approach where the default logging level remains
> > > at DEBUG while re-categorisation occurs would mean that nothing
> > > materially changes until we are happy with the categorisations.
> > > Then we can default the logging level to INFO.
> > >
> >
> > I created a issue for this a while ago:
> > https://issues.apache.org/jira/browse/CLOUDSTACK-8645
> >
> > Short: Yes. Just change this where needed.
> >
> > Wido
> >
> > >
> > >
> > > Thoughts everyone?
> > >
> > >
> > >
> > >
> > > Kind regards,
> > >
> > > Paul Angus
> > >
> > >
> > > Regards,
> > >
> > > Paul Angus
> > >
> > > paul.angus@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
> >
>
>
>
> --
> Daan
>
Re: [discuss] CloudStack logging
Posted by Pierre-Luc Dion <pd...@cloudops.com>.
Something that could be interesting also is to remove VR and systemVMs logs
from the cloudstack-management.log and have them into a separate files. I'm
still not sure of those logs are traps only when there is problems, if it's
the case it make sense to keep them in cloudstack-management.log.
Anyhow, it might be interesting to collect VR and systemVMs logs from the
management servers.
The log structure is also pretty stable so far which make integration with
ELK fairly easy which is good!
What about the catalina.out? is there any way to reduce is size?
I'll update the JIRA issue with problematic troubleshooting logs that are
not clear to interpret...
Thanks Paul for taking that one !
PL
On Fri, Apr 15, 2016 at 4:44 AM, Daan Hoogland <da...@gmail.com>
wrote:
> Paul,
>
> I think that asking for input is a good approach. Those that have problem
> with a certain log message or - sequence are the best to say what should be
> different.
>
> Whether we should make hundreds of PRs or one?.. neither. I would say
> change anything from a big class to a small package, or get the flow for
> one API call and change anything in that. That would be cross package and
> may require classes in the management -, service -, orchestration and
> resource layers to be revisited for several flows.
>
>
> On Fri, Apr 15, 2016 at 10:06 AM, Paul Angus <pa...@shapeblue.com>
> wrote:
>
> > Great, I'll pick up the issue that Wido has already created
> > https://issues.apache.org/jira/browse/CLOUDSTACK-8645
> >
> > Does anyone have an suggestions wrt to structure of pull requests?
> > I'm inclinded to ask the community for copies of logs - particularly ones
> > that people have found difficult to read so that I or others can work
> > through them looking for anamolies - it's very messy from a thoroughness
> > point of view - but it would return the most useful changes first. It
> also
> > puts the error msg in a context to make the categorisation more acurate.
> >
> >
> >
> > Kind regards,
> >
> > Paul Angus
> >
> > Regards,
> >
> > Paul Angus
> >
> > paul.angus@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> > @shapeblue
> >
> > -----Original Message-----
> > From: Wido den Hollander [mailto:wido@widodh.nl]
> > Sent: 14 April 2016 15:04
> > To: Paul Angus <pa...@shapeblue.com>; dev@cloudstack.apache.org
> > Subject: Re: [discuss] CloudStack logging
> >
> >
> > > Op 14 april 2016 om 14:49 schreef Paul Angus <paul.angus@shapeblue.com
> >:
> > >
> > >
> > > Hi All
> > >
> > > I think that we can all agree that CloudStack logs are very difficult
> > > to read, especially for operational staff.
> > >
> > > I believe that the primary reason for this is that a large amount of
> > > what should be INFO level events are categorised as DEBUG. The
> > > logging therefore must be set at DEBUG for the logs to capture these
> > > events. By defaulting the logging to DEBUG we include a lot of detail
> > > which is counter-productive when using the logs to diagnose operational
> > issues.
> > >
> >
> > Yes!
> >
> > > I think the situation can be greatly improved by reviewing the
> > > categorisation of logged events and where necessary re-categorise then
> > > such that INFO, WARN and ERROR logging would be sufficient for an
> > 'operational' admin.
> > >
> > > I think a phased approach where the default logging level remains at
> > > DEBUG while re-categorisation occurs would mean that nothing
> > > materially changes until we are happy with the categorisations. Then
> > > we can default the logging level to INFO.
> > >
> >
> > I created a issue for this a while ago:
> > https://issues.apache.org/jira/browse/CLOUDSTACK-8645
> >
> > Short: Yes. Just change this where needed.
> >
> > Wido
> >
> > >
> > >
> > > Thoughts everyone?
> > >
> > >
> > >
> > >
> > > Kind regards,
> > >
> > > Paul Angus
> > >
> > >
> > > Regards,
> > >
> > > Paul Angus
> > >
> > > paul.angus@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
> >
>
>
>
> --
> Daan
>
Re: [discuss] CloudStack logging
Posted by Daan Hoogland <da...@gmail.com>.
Paul,
I think that asking for input is a good approach. Those that have problem
with a certain log message or - sequence are the best to say what should be
different.
Whether we should make hundreds of PRs or one?.. neither. I would say
change anything from a big class to a small package, or get the flow for
one API call and change anything in that. That would be cross package and
may require classes in the management -, service -, orchestration and
resource layers to be revisited for several flows.
On Fri, Apr 15, 2016 at 10:06 AM, Paul Angus <pa...@shapeblue.com>
wrote:
> Great, I'll pick up the issue that Wido has already created
> https://issues.apache.org/jira/browse/CLOUDSTACK-8645
>
> Does anyone have an suggestions wrt to structure of pull requests?
> I'm inclinded to ask the community for copies of logs - particularly ones
> that people have found difficult to read so that I or others can work
> through them looking for anamolies - it's very messy from a thoroughness
> point of view - but it would return the most useful changes first. It also
> puts the error msg in a context to make the categorisation more acurate.
>
>
>
> Kind regards,
>
> Paul Angus
>
> Regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
> -----Original Message-----
> From: Wido den Hollander [mailto:wido@widodh.nl]
> Sent: 14 April 2016 15:04
> To: Paul Angus <pa...@shapeblue.com>; dev@cloudstack.apache.org
> Subject: Re: [discuss] CloudStack logging
>
>
> > Op 14 april 2016 om 14:49 schreef Paul Angus <pa...@shapeblue.com>:
> >
> >
> > Hi All
> >
> > I think that we can all agree that CloudStack logs are very difficult
> > to read, especially for operational staff.
> >
> > I believe that the primary reason for this is that a large amount of
> > what should be INFO level events are categorised as DEBUG. The
> > logging therefore must be set at DEBUG for the logs to capture these
> > events. By defaulting the logging to DEBUG we include a lot of detail
> > which is counter-productive when using the logs to diagnose operational
> issues.
> >
>
> Yes!
>
> > I think the situation can be greatly improved by reviewing the
> > categorisation of logged events and where necessary re-categorise then
> > such that INFO, WARN and ERROR logging would be sufficient for an
> 'operational' admin.
> >
> > I think a phased approach where the default logging level remains at
> > DEBUG while re-categorisation occurs would mean that nothing
> > materially changes until we are happy with the categorisations. Then
> > we can default the logging level to INFO.
> >
>
> I created a issue for this a while ago:
> https://issues.apache.org/jira/browse/CLOUDSTACK-8645
>
> Short: Yes. Just change this where needed.
>
> Wido
>
> >
> >
> > Thoughts everyone?
> >
> >
> >
> >
> > Kind regards,
> >
> > Paul Angus
> >
> >
> > Regards,
> >
> > Paul Angus
> >
> > paul.angus@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
>
--
Daan
RE: [discuss] CloudStack logging
Posted by Paul Angus <pa...@shapeblue.com>.
Great, I'll pick up the issue that Wido has already created
https://issues.apache.org/jira/browse/CLOUDSTACK-8645
Does anyone have an suggestions wrt to structure of pull requests?
I'm inclinded to ask the community for copies of logs - particularly ones that people have found difficult to read so that I or others can work through them looking for anamolies - it's very messy from a thoroughness point of view - but it would return the most useful changes first. It also puts the error msg in a context to make the categorisation more acurate.
Kind regards,
Paul Angus
Regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
-----Original Message-----
From: Wido den Hollander [mailto:wido@widodh.nl]
Sent: 14 April 2016 15:04
To: Paul Angus <pa...@shapeblue.com>; dev@cloudstack.apache.org
Subject: Re: [discuss] CloudStack logging
> Op 14 april 2016 om 14:49 schreef Paul Angus <pa...@shapeblue.com>:
>
>
> Hi All
>
> I think that we can all agree that CloudStack logs are very difficult
> to read, especially for operational staff.
>
> I believe that the primary reason for this is that a large amount of
> what should be INFO level events are categorised as DEBUG. The
> logging therefore must be set at DEBUG for the logs to capture these
> events. By defaulting the logging to DEBUG we include a lot of detail
> which is counter-productive when using the logs to diagnose operational issues.
>
Yes!
> I think the situation can be greatly improved by reviewing the
> categorisation of logged events and where necessary re-categorise then
> such that INFO, WARN and ERROR logging would be sufficient for an 'operational' admin.
>
> I think a phased approach where the default logging level remains at
> DEBUG while re-categorisation occurs would mean that nothing
> materially changes until we are happy with the categorisations. Then
> we can default the logging level to INFO.
>
I created a issue for this a while ago:
https://issues.apache.org/jira/browse/CLOUDSTACK-8645
Short: Yes. Just change this where needed.
Wido
>
>
> Thoughts everyone?
>
>
>
>
> Kind regards,
>
> Paul Angus
>
>
> Regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
Re: [discuss] CloudStack logging
Posted by Wido den Hollander <wi...@widodh.nl>.
> Op 14 april 2016 om 14:49 schreef Paul Angus <pa...@shapeblue.com>:
>
>
> Hi All
>
> I think that we can all agree that CloudStack logs are very difficult to read,
> especially for operational staff.
>
> I believe that the primary reason for this is that a large amount of what
> should be INFO level events are categorised as DEBUG. The logging therefore
> must be set at DEBUG for the logs to capture these events. By defaulting the
> logging to DEBUG we include a lot of detail which is counter-productive when
> using the logs to diagnose operational issues.
>
Yes!
> I think the situation can be greatly improved by reviewing the categorisation
> of logged events and where necessary re-categorise then such that INFO, WARN
> and ERROR logging would be sufficient for an 'operational' admin.
>
> I think a phased approach where the default logging level remains at DEBUG
> while re-categorisation occurs would mean that nothing materially changes
> until we are happy with the categorisations. Then we can default the logging
> level to INFO.
>
I created a issue for this a while ago:
https://issues.apache.org/jira/browse/CLOUDSTACK-8645
Short: Yes. Just change this where needed.
Wido
>
>
> Thoughts everyone?
>
>
>
>
> Kind regards,
>
> Paul Angus
>
>
> Regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue