You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Cory Fuchs <CF...@zimcom.net> on 2016/06/17 15:38:18 UTC
Help with Usage data - There is no data from Cloudmonkey of types:
4,5,10,21,22,23 or 24 (Network and VM Disk data)
Hello Everyone,
I hope that someone can point me in the direction of how to resolve my issue and let me know what I am doing wrong.
Cloudstack version: 4.5.2
Running the following command from Cloudmonkey returns no results.
list usagerecords startdate=2016-06-16 enddate=2016-06-17 type=4
Changing the type to any of the following returns no results: 4, 5, 10, 21, 22, or 23
Changing the type to any of the following does return results: 1, 2, 3, 6, 7, 8, 9, 11, 12, 13, 14, or 25
This tells me the usage job is running and collecting data. Just not on the Network or VM Disk data.
Looking at /var/log/cloudstack/usage/usage.log
Grepping for an accountId that I know has usage on. I get lines like the following.
2016-06-17 00:15:03,347 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) creating networkHelperEntry...accountId: 7 in zone: 1; abr: 142919240951; abs: 40911793471; curABS: 40911793471; curABR: 142919240951; ubs: 0; ubr: 0
2016-06-17 00:15:03,348 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) getting current accounted bytes for... accountId: 7 in zone: 1; abr: 39361808078; abs: 1006529247
2016-06-17 00:15:03,348 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) creating networkHelperEntry... accountId: 7 in zone: 1; abr: 39361808078; abs: 1006529247; curABS: 1006529247; curABR: 39361808078; ubs: 0; ubr: 0
Grepping for account 7. I get some of the following lines:
2016-06-17 00:15:13,976 DEBUG [usage.parser.NetworkUsageParser] (Usage-Job-1:null) No usage record (0 bytes used) generated for account: 7
2016-06-17 00:15:13,976 DEBUG [usage.parser.NetworkUsageParser] (Usage-Job-1:null) No usage record (0 bytes used) generated for account: 7
2016-06-17 00:15:13,976 DEBUG [usage.parser.VmDiskUsageParser] (Usage-Job-1:null) Parsing all Vm Disk usage events for account: 7
2016-06-17 00:15:14,074 DEBUG [usage.parser.VmDiskUsageParser] (Usage-Job-1:null) No vm disk usage record (0 bytes used) generated for account: 7
2016-06-17 00:15:14,074 DEBUG [usage.parser.VmDiskUsageParser] (Usage-Job-1:null) No vm disk usage record (0 bytes used) generated for account: 7
Where can I look or what am I doing wrong?
I am mainly looking for network usage.
Thank you to anyone that can assist.
Cory Fuchs
Re: Help with Usage data - There is no data from Cloudmonkey of
types: 4,5,10,21,22,23 or 24 (Network and VM Disk data)
Posted by Rajani Karuturi <ra...@apache.org>.
I think it should go to usage_network table.
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Usage+and+Usage+Events
https://github.com/apache/cloudstack/blob/master/usage/src/com/cloud/usage/UsageManagerImpl.java#L668
~Rajani
On Wed, Jun 22, 2016 at 7:38 PM, Cory Fuchs <CF...@zimcom.net> wrote:
> As an update if anyone can offer any assistance.
>
> The data is flowing into cloud_usage.user_statistics. Data is updating
> after the usage timer runs.
>
> Where is it supposed to flow from there?
>
> Cory Fuchs
>
>
> ________________________________________
> From: Cory Fuchs <CF...@zimcom.net>
> Sent: Tuesday, June 21, 2016 9:22 AM
> To: users@cloudstack.apache.org
> Subject: Re: Help with Usage data - There is no data from Cloudmonkey of
> types: 4,5,10,21,22,23 or 24 (Network and VM Disk data)
>
> Hello Ilya,
>
> I have not found a resolution yet. Any help would be greatly appreciated.
>
> I have been going through logs and trying to pin down where it is
> failing. However, without knowing the workflow of the data I have been
> shooting in the dark. This is what I have found so far.
>
> From the /var/log/cloudstack/management/management-server.log file
> I am seeing entries like this. Telling me that the management server is
> doing the collecting of the data.
>
> 2016-06-21 02:49:03,423 DEBUG [c.c.a.t.Request]
> (DirectAgent-189:ctx-49882d6b) Seq 11-6346697774871958512: Processing: {
> Ans: , MgmtId: 345049798930, via: 11, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-610-VM","bytesSent":1139691977597,"bytesReceived":163740483889,"result":true,"details":"success","wait":0}}]
> }
>
>
> If I query cloud.user_statistics table for device_id 610 (i.e. r-610-vm
> matches the device_id 610).
> The following entries are increasing.
> current_bytes_received: 163614052302
> current_bytes_sent: 1137990941434
> agg_bytes_received: 8817678036
> agg_bytes_sent: 33498435712
>
> I just started tracking the values in the cloud_usage.user_statistics
> table for device_id 610. However I didn't track it until after our
> usage.stats.job.execution had already ran and it will not run again for
> another 11 hours. So I don't know if the cloud_usage.user_statistics is
> getting updated.
>
> Are there any other data points that I can track to help troubleshoot this?
> Am I looking in the wrong places?
>
> Any information you can provide would be appreciated.
>
> Cory Fuchs
>
>
> ________________________________________
> From: ilya <il...@gmail.com>
> Sent: Tuesday, June 21, 2016 1:57 AM
> To: users@cloudstack.apache.org
> Subject: Re: Help with Usage data - There is no data from Cloudmonkey of
> types: 4,5,10,21,22,23 or 24 (Network and VM Disk data)
>
> Cory
>
> Just curious if you made any headway on this issue or still need help.
>
> Regards
> ilya
>
> On 6/17/16 8:38 AM, Cory Fuchs wrote:
> > Hello Everyone,
> >
> >
> > I hope that someone can point me in the direction of how to resolve my
> issue and let me know what I am doing wrong.
> >
> >
> > Cloudstack version: 4.5.2
> >
> >
> > Running the following command from Cloudmonkey returns no results.
> >
> >
> > list usagerecords startdate=2016-06-16 enddate=2016-06-17 type=4
> >
> >
> > Changing the type to any of the following returns no results: 4, 5, 10,
> 21, 22, or 23
> >
> > Changing the type to any of the following does return results: 1, 2, 3,
> 6, 7, 8, 9, 11, 12, 13, 14, or 25
> >
> >
> > This tells me the usage job is running and collecting data. Just not on
> the Network or VM Disk data.
> >
> >
> > Looking at /var/log/cloudstack/usage/usage.log
> >
> >
> > Grepping for an accountId that I know has usage on. I get lines like
> the following.
> >
> >
> > 2016-06-17 00:15:03,347 DEBUG [cloud.usage.UsageManagerImpl]
> (Usage-Job-1:null) creating networkHelperEntry...accountId: 7 in zone: 1;
> abr: 142919240951; abs: 40911793471; curABS: 40911793471; curABR:
> 142919240951; ubs: 0; ubr: 0
> >
> > 2016-06-17 00:15:03,348 DEBUG [cloud.usage.UsageManagerImpl]
> (Usage-Job-1:null) getting current accounted bytes for... accountId: 7 in
> zone: 1; abr: 39361808078; abs: 1006529247
> >
> > 2016-06-17 00:15:03,348 DEBUG [cloud.usage.UsageManagerImpl]
> (Usage-Job-1:null) creating networkHelperEntry... accountId: 7 in zone: 1;
> abr: 39361808078; abs: 1006529247; curABS: 1006529247; curABR: 39361808078;
> ubs: 0; ubr: 0
> >
> >
> >
> > Grepping for account 7. I get some of the following lines:
> >
> >
> > 2016-06-17 00:15:13,976 DEBUG [usage.parser.NetworkUsageParser]
> (Usage-Job-1:null) No usage record (0 bytes used) generated for account: 7
> >
> > 2016-06-17 00:15:13,976 DEBUG [usage.parser.NetworkUsageParser]
> (Usage-Job-1:null) No usage record (0 bytes used) generated for account: 7
> >
> > 2016-06-17 00:15:13,976 DEBUG [usage.parser.VmDiskUsageParser]
> (Usage-Job-1:null) Parsing all Vm Disk usage events for account: 7
> >
> > 2016-06-17 00:15:14,074 DEBUG [usage.parser.VmDiskUsageParser]
> (Usage-Job-1:null) No vm disk usage record (0 bytes used) generated for
> account: 7
> >
> > 2016-06-17 00:15:14,074 DEBUG [usage.parser.VmDiskUsageParser]
> (Usage-Job-1:null) No vm disk usage record (0 bytes used) generated for
> account: 7
> >
> >
> > Where can I look or what am I doing wrong?
> >
> > I am mainly looking for network usage.
> >
> >
> > Thank you to anyone that can assist.
> >
> >
> > Cory Fuchs
> >
> >
> >
>
Re: Help with Usage data - There is no data from Cloudmonkey of
types: 4,5,10,21,22,23 or 24 (Network and VM Disk data)
Posted by Cory Fuchs <CF...@zimcom.net>.
As an update if anyone can offer any assistance.
The data is flowing into cloud_usage.user_statistics. Data is updating after the usage timer runs.
Where is it supposed to flow from there?
Cory Fuchs
________________________________________
From: Cory Fuchs <CF...@zimcom.net>
Sent: Tuesday, June 21, 2016 9:22 AM
To: users@cloudstack.apache.org
Subject: Re: Help with Usage data - There is no data from Cloudmonkey of types: 4,5,10,21,22,23 or 24 (Network and VM Disk data)
Hello Ilya,
I have not found a resolution yet. Any help would be greatly appreciated.
I have been going through logs and trying to pin down where it is failing. However, without knowing the workflow of the data I have been shooting in the dark. This is what I have found so far.
From the /var/log/cloudstack/management/management-server.log file
I am seeing entries like this. Telling me that the management server is doing the collecting of the data.
2016-06-21 02:49:03,423 DEBUG [c.c.a.t.Request] (DirectAgent-189:ctx-49882d6b) Seq 11-6346697774871958512: Processing: { Ans: , MgmtId: 345049798930, via: 11, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-610-VM","bytesSent":1139691977597,"bytesReceived":163740483889,"result":true,"details":"success","wait":0}}] }
If I query cloud.user_statistics table for device_id 610 (i.e. r-610-vm matches the device_id 610).
The following entries are increasing.
current_bytes_received: 163614052302
current_bytes_sent: 1137990941434
agg_bytes_received: 8817678036
agg_bytes_sent: 33498435712
I just started tracking the values in the cloud_usage.user_statistics table for device_id 610. However I didn't track it until after our usage.stats.job.execution had already ran and it will not run again for another 11 hours. So I don't know if the cloud_usage.user_statistics is getting updated.
Are there any other data points that I can track to help troubleshoot this?
Am I looking in the wrong places?
Any information you can provide would be appreciated.
Cory Fuchs
________________________________________
From: ilya <il...@gmail.com>
Sent: Tuesday, June 21, 2016 1:57 AM
To: users@cloudstack.apache.org
Subject: Re: Help with Usage data - There is no data from Cloudmonkey of types: 4,5,10,21,22,23 or 24 (Network and VM Disk data)
Cory
Just curious if you made any headway on this issue or still need help.
Regards
ilya
On 6/17/16 8:38 AM, Cory Fuchs wrote:
> Hello Everyone,
>
>
> I hope that someone can point me in the direction of how to resolve my issue and let me know what I am doing wrong.
>
>
> Cloudstack version: 4.5.2
>
>
> Running the following command from Cloudmonkey returns no results.
>
>
> list usagerecords startdate=2016-06-16 enddate=2016-06-17 type=4
>
>
> Changing the type to any of the following returns no results: 4, 5, 10, 21, 22, or 23
>
> Changing the type to any of the following does return results: 1, 2, 3, 6, 7, 8, 9, 11, 12, 13, 14, or 25
>
>
> This tells me the usage job is running and collecting data. Just not on the Network or VM Disk data.
>
>
> Looking at /var/log/cloudstack/usage/usage.log
>
>
> Grepping for an accountId that I know has usage on. I get lines like the following.
>
>
> 2016-06-17 00:15:03,347 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) creating networkHelperEntry...accountId: 7 in zone: 1; abr: 142919240951; abs: 40911793471; curABS: 40911793471; curABR: 142919240951; ubs: 0; ubr: 0
>
> 2016-06-17 00:15:03,348 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) getting current accounted bytes for... accountId: 7 in zone: 1; abr: 39361808078; abs: 1006529247
>
> 2016-06-17 00:15:03,348 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) creating networkHelperEntry... accountId: 7 in zone: 1; abr: 39361808078; abs: 1006529247; curABS: 1006529247; curABR: 39361808078; ubs: 0; ubr: 0
>
>
>
> Grepping for account 7. I get some of the following lines:
>
>
> 2016-06-17 00:15:13,976 DEBUG [usage.parser.NetworkUsageParser] (Usage-Job-1:null) No usage record (0 bytes used) generated for account: 7
>
> 2016-06-17 00:15:13,976 DEBUG [usage.parser.NetworkUsageParser] (Usage-Job-1:null) No usage record (0 bytes used) generated for account: 7
>
> 2016-06-17 00:15:13,976 DEBUG [usage.parser.VmDiskUsageParser] (Usage-Job-1:null) Parsing all Vm Disk usage events for account: 7
>
> 2016-06-17 00:15:14,074 DEBUG [usage.parser.VmDiskUsageParser] (Usage-Job-1:null) No vm disk usage record (0 bytes used) generated for account: 7
>
> 2016-06-17 00:15:14,074 DEBUG [usage.parser.VmDiskUsageParser] (Usage-Job-1:null) No vm disk usage record (0 bytes used) generated for account: 7
>
>
> Where can I look or what am I doing wrong?
>
> I am mainly looking for network usage.
>
>
> Thank you to anyone that can assist.
>
>
> Cory Fuchs
>
>
>
Re: Help with Usage data - There is no data from Cloudmonkey of
types: 4,5,10,21,22,23 or 24 (Network and VM Disk data)
Posted by Cory Fuchs <CF...@zimcom.net>.
Hello Ilya,
I have not found a resolution yet. Any help would be greatly appreciated.
I have been going through logs and trying to pin down where it is failing. However, without knowing the workflow of the data I have been shooting in the dark. This is what I have found so far.
From the /var/log/cloudstack/management/management-server.log file
I am seeing entries like this. Telling me that the management server is doing the collecting of the data.
2016-06-21 02:49:03,423 DEBUG [c.c.a.t.Request] (DirectAgent-189:ctx-49882d6b) Seq 11-6346697774871958512: Processing: { Ans: , MgmtId: 345049798930, via: 11, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-610-VM","bytesSent":1139691977597,"bytesReceived":163740483889,"result":true,"details":"success","wait":0}}] }
If I query cloud.user_statistics table for device_id 610 (i.e. r-610-vm matches the device_id 610).
The following entries are increasing.
current_bytes_received: 163614052302
current_bytes_sent: 1137990941434
agg_bytes_received: 8817678036
agg_bytes_sent: 33498435712
I just started tracking the values in the cloud_usage.user_statistics table for device_id 610. However I didn't track it until after our usage.stats.job.execution had already ran and it will not run again for another 11 hours. So I don't know if the cloud_usage.user_statistics is getting updated.
Are there any other data points that I can track to help troubleshoot this?
Am I looking in the wrong places?
Any information you can provide would be appreciated.
Cory Fuchs
________________________________________
From: ilya <il...@gmail.com>
Sent: Tuesday, June 21, 2016 1:57 AM
To: users@cloudstack.apache.org
Subject: Re: Help with Usage data - There is no data from Cloudmonkey of types: 4,5,10,21,22,23 or 24 (Network and VM Disk data)
Cory
Just curious if you made any headway on this issue or still need help.
Regards
ilya
On 6/17/16 8:38 AM, Cory Fuchs wrote:
> Hello Everyone,
>
>
> I hope that someone can point me in the direction of how to resolve my issue and let me know what I am doing wrong.
>
>
> Cloudstack version: 4.5.2
>
>
> Running the following command from Cloudmonkey returns no results.
>
>
> list usagerecords startdate=2016-06-16 enddate=2016-06-17 type=4
>
>
> Changing the type to any of the following returns no results: 4, 5, 10, 21, 22, or 23
>
> Changing the type to any of the following does return results: 1, 2, 3, 6, 7, 8, 9, 11, 12, 13, 14, or 25
>
>
> This tells me the usage job is running and collecting data. Just not on the Network or VM Disk data.
>
>
> Looking at /var/log/cloudstack/usage/usage.log
>
>
> Grepping for an accountId that I know has usage on. I get lines like the following.
>
>
> 2016-06-17 00:15:03,347 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) creating networkHelperEntry...accountId: 7 in zone: 1; abr: 142919240951; abs: 40911793471; curABS: 40911793471; curABR: 142919240951; ubs: 0; ubr: 0
>
> 2016-06-17 00:15:03,348 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) getting current accounted bytes for... accountId: 7 in zone: 1; abr: 39361808078; abs: 1006529247
>
> 2016-06-17 00:15:03,348 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) creating networkHelperEntry... accountId: 7 in zone: 1; abr: 39361808078; abs: 1006529247; curABS: 1006529247; curABR: 39361808078; ubs: 0; ubr: 0
>
>
>
> Grepping for account 7. I get some of the following lines:
>
>
> 2016-06-17 00:15:13,976 DEBUG [usage.parser.NetworkUsageParser] (Usage-Job-1:null) No usage record (0 bytes used) generated for account: 7
>
> 2016-06-17 00:15:13,976 DEBUG [usage.parser.NetworkUsageParser] (Usage-Job-1:null) No usage record (0 bytes used) generated for account: 7
>
> 2016-06-17 00:15:13,976 DEBUG [usage.parser.VmDiskUsageParser] (Usage-Job-1:null) Parsing all Vm Disk usage events for account: 7
>
> 2016-06-17 00:15:14,074 DEBUG [usage.parser.VmDiskUsageParser] (Usage-Job-1:null) No vm disk usage record (0 bytes used) generated for account: 7
>
> 2016-06-17 00:15:14,074 DEBUG [usage.parser.VmDiskUsageParser] (Usage-Job-1:null) No vm disk usage record (0 bytes used) generated for account: 7
>
>
> Where can I look or what am I doing wrong?
>
> I am mainly looking for network usage.
>
>
> Thank you to anyone that can assist.
>
>
> Cory Fuchs
>
>
>
Re: Help with Usage data - There is no data from Cloudmonkey of
types: 4,5,10,21,22,23 or 24 (Network and VM Disk data)
Posted by ilya <il...@gmail.com>.
Cory
Just curious if you made any headway on this issue or still need help.
Regards
ilya
On 6/17/16 8:38 AM, Cory Fuchs wrote:
> Hello Everyone,
>
>
> I hope that someone can point me in the direction of how to resolve my issue and let me know what I am doing wrong.
>
>
> Cloudstack version: 4.5.2
>
>
> Running the following command from Cloudmonkey returns no results.
>
>
> list usagerecords startdate=2016-06-16 enddate=2016-06-17 type=4
>
>
> Changing the type to any of the following returns no results: 4, 5, 10, 21, 22, or 23
>
> Changing the type to any of the following does return results: 1, 2, 3, 6, 7, 8, 9, 11, 12, 13, 14, or 25
>
>
> This tells me the usage job is running and collecting data. Just not on the Network or VM Disk data.
>
>
> Looking at /var/log/cloudstack/usage/usage.log
>
>
> Grepping for an accountId that I know has usage on. I get lines like the following.
>
>
> 2016-06-17 00:15:03,347 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) creating networkHelperEntry...accountId: 7 in zone: 1; abr: 142919240951; abs: 40911793471; curABS: 40911793471; curABR: 142919240951; ubs: 0; ubr: 0
>
> 2016-06-17 00:15:03,348 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) getting current accounted bytes for... accountId: 7 in zone: 1; abr: 39361808078; abs: 1006529247
>
> 2016-06-17 00:15:03,348 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) creating networkHelperEntry... accountId: 7 in zone: 1; abr: 39361808078; abs: 1006529247; curABS: 1006529247; curABR: 39361808078; ubs: 0; ubr: 0
>
>
>
> Grepping for account 7. I get some of the following lines:
>
>
> 2016-06-17 00:15:13,976 DEBUG [usage.parser.NetworkUsageParser] (Usage-Job-1:null) No usage record (0 bytes used) generated for account: 7
>
> 2016-06-17 00:15:13,976 DEBUG [usage.parser.NetworkUsageParser] (Usage-Job-1:null) No usage record (0 bytes used) generated for account: 7
>
> 2016-06-17 00:15:13,976 DEBUG [usage.parser.VmDiskUsageParser] (Usage-Job-1:null) Parsing all Vm Disk usage events for account: 7
>
> 2016-06-17 00:15:14,074 DEBUG [usage.parser.VmDiskUsageParser] (Usage-Job-1:null) No vm disk usage record (0 bytes used) generated for account: 7
>
> 2016-06-17 00:15:14,074 DEBUG [usage.parser.VmDiskUsageParser] (Usage-Job-1:null) No vm disk usage record (0 bytes used) generated for account: 7
>
>
> Where can I look or what am I doing wrong?
>
> I am mainly looking for network usage.
>
>
> Thank you to anyone that can assist.
>
>
> Cory Fuchs
>
>
>