You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Wei Zhou (JIRA)" <ji...@apache.org> on 2013/06/11 08:50:20 UTC
[jira] [Closed] (CLOUDSTACK-685) CloudStack 4.0 Network Usage is
ZERO
[ https://issues.apache.org/jira/browse/CLOUDSTACK-685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wei Zhou closed CLOUDSTACK-685.
-------------------------------
> CloudStack 4.0 Network Usage is ZERO
> ------------------------------------
>
> Key: CLOUDSTACK-685
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-685
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: Usage
> Affects Versions: 4.0.0
> Reporter: Wei Zhou
> Assignee: Kishan Kavala
> Fix For: 4.0.1
>
>
> Problem description:
> The usage server can give the system usage of each virtual machine (such as running time, ServiceOffering, IPAddress, Volume, Template, ISO, Port Forwarding Rule, Network offering), except the network bytes sent/received. This problem only exists in CloudStack 4.0. In CloudStack 3.0, the usage server works well.
> We looked into Iptables of VR, its working fine (look at the output of 10g download). This is just a sample.
> root@r-17-VM:/# iptables -nvx -L NETWORK_STATS
> Chain NETWORK_STATS (3 references)
> pkts bytes target prot opt in out source destination
> 246477 12943171 all -- eth0 eth2 0.0.0.0/0 0.0.0.0/0
> 125789 1008395759 all -- eth2 eth0 0.0.0.0/0 0.0.0.0/0
> 0 0 tcp -- !eth0 eth2 0.0.0.0/0 0.0.0.0/0
> 0 0 tcp -- eth2 !eth0 0.0.0.0/0 0.0.0.0/0
> We tried to debug it further & found the following:
> (1) Checked the CloudStack logs /var/log/cloud/management/management-server.log
> 2012-12-18 15:05:51,130 DEBUG [agent.transport.Request] (AgentManager-Handler-8:null) Seq 1-1158742136: Processing: { Ans: , MgmtId: 345051509349, via: 1, Ver: v1, Flags: 10, [{"NetworkUsageAnswer":{"routerName":"r-4-VM","bytesSent":5928,"bytesReceived":6188,"result":true,"details":"","wait":0}}] }
> 2012-12-18 15:05:51,130 DEBUG [agent.transport.Request] (RouterMonitor-1:null) Seq 1-1158742136: Received: { Ans: , MgmtId: 345051509349, via: 1, Ver: v1, Flags: 10, { NetworkUsageAnswer } }
> 2012-12-18 15:05:51,130 DEBUG [agent.manager.AgentManagerImpl] (RouterMonitor-1:null) Details from executing class com.cloud.agent.api.NetworkUsageCommand:
> 2012-12-18 15:05:51,131 WARN [network.router.VirtualNetworkApplianceManagerImpl] (RouterMonitor-1:null) unable to find stats for account: 2
> We can see that AgentManager works well. It can get the network usage.
> (2) Checked the the database.
> There are 4 tables, cloud.user_statistics, cloud_usage.user_statistics, cloud_usage.usage_network and cloud_usage.cloud_usage which is used by API & we're trying to filter Network Usage using 'usage_type' 4 or 5
> As we add new network (VR), it adds a row in cloud.user_statistics & eventually ends up (could be because of generateUsageRecords API command) in other 2 tables ( cloud_usage.user_statistics, cloud_usage.usage_network )
> mysql> select * from cloud.user_statistics;
> +----+----------------+------------+-------------------+-----------+--------------+------------+--------------------+----------------+------------------------+--------
> | id | data_center_id | account_id | public_ip_address | device_id | device_type | network_id | net_bytes_received | net_bytes_sent | current_bytes_received | current
> +----+----------------+------------+-------------------+-----------+--------------+------------+--------------------+----------------+------------------------+--------
> | 1 | 1 | 2 | NULL | 4 | DomainRouter | 204 | 0 | 0 | 0 |
> | 2 | 1 | 2 | NULL | 6 | DomainRouter | 205 | 0 | 0 | 0 |
> +----+----------------+------------+-------------------+-----------+--------------+------------+--------------------+----------------+------------------------+--------
> (3) Checked the source code.
> (a) /cloud-server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java ( I have deleted some unrelated source codes)
> public void run() {
> for (DomainRouterVO router : routers) {
> String privateIP = router.getPrivateIpAddress(); // from domain_router table (id=4, name=r-4-VM, vm_instance.private_ip_address=169.254.1.17)
> if (privateIP != null) {
> List<? extends Nic> routerNics = _nicDao.listByVmId(router.getId()); // from nics table (instance_id=4). Found 3 records. Return (networkid= 204/202/200)
> for (Nic routerNic : routerNics) {
> Network network = _networkMgr.getNetwork(routerNic.getNetworkId()); // from nics table.
> if (network.getTrafficType() == TrafficType.Public) { // get traffic_type in networks table. Return Guest/Control/Public.
> UserStatisticsVO previousStats = _statsDao.findBy(router.getAccountId(),
> router.getDataCenterIdToDeployIn(), network.getId(), null, router.getId(), router.getType().toString());
> if (answer != null) {
> Transaction txn = Transaction.open(Transaction.CLOUD_DB);
> try {
> txn.start();
> UserStatisticsVO stats = _statsDao.lock(router.getAccountId(),
> router.getDataCenterIdToDeployIn(), network.getId(), routerNic.getIp4Address(), router.getId(), router.getType().toString()); // the parameter is “2, 1, 200, 10.11.102.168, 4”. It’s inconsistent to database.
> if (stats == null) {
> s_logger.warn("unable to find stats for account: " + router.getAccountId());
> continue;
> }
> ………………..
> }
> (b) /cloud-server/src/com/cloud/vm/dao/DomainRouterDaoImpl.java
> public void addRouterToGuestNetwork(VirtualRouter router, Network guestNetwork) {
> if (stats == null) {
> stats = new UserStatisticsVO(router.getAccountId(), router.getDataCenterIdToDeployIn(), null, router.getId(),router.getType().toString(), guestNetwork.getId()); // the parameter is “2, 1, null, 204, 4”. It’s consistent to database.
> _userStatsDao.persist(stats);
> }
> }
> (4) Changed databases and tested.
> Since the IP Address is null & the network_id is not public in the cloud.user_statistics table, it's not doing anything related to traffic. As a result all the column related to traffic stats remains 0.
> But if we explicitly add a row in cloud.user_statistics with network's (VR) Source NATed IP Address and the public network_id, It worked!
> | id | data_center_id | account_id | public_ip_address | device_id | device_type | network_id |
> | 3 | 1 | 2 | 10.11.102.168 | 4 | DomainRouter | 200 |
> After 5 minutes, we can get the VR data traffic.It looks like
> | id | data_center_id | account_id | public_ip_address | device_id | device_type | network_id | net_bytes_received | net_bytes_sent | current_bytes_received | current_bytes_sent | agg_bytes_received | agg_bytes_sent |
> | 3 | 1 | 2 | 10.11.102.168 | 4 | DomainRouter | 200 | 0 | 0 | 2284239008 | 33613977 | 2284239008 | 33613977 |
> From Step 1 to Step 4, we can see that maybe the inconsistent of (3.a) and (3.b) is the root cause of this problem.
> We notice that there are some changes of VirtualNetworkApplianceManagerImpl.java between CloudStack 3.0.2 and CloudStack 4.0.0. This is why this problem only exists in CloudStack 4.0.0.
> In CloudStack 4.0.0 : https://github.com/CloudStack-extras/CloudStack-archive/blob/master/server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
> In CloudStack 3.0.2 : https://github.com/CloudStack-extras/CloudStack-archive/blob/3.0.2.maintenance/server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
> Why does CloudStack make this change which cause this problem?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira