You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Fariborz Navidan <md...@gmail.com> on 2021/05/19 22:04:16 UTC
Change host IP
Hello,
We have migrated our server and as a result, we need to change the IP
address. I have modified agent.properties and all ip address fields of host
record to match the new IP address. But unfortunately, management server is
unable to connect to it. I forced the host to go for maintenance checking
"force". Now when I disable maintenance mode I get error " Failed to
prepare host for maintenance due to: Unable to prepare for maintenance host
1".
Bellow is the agent log:
2021-05-19 23:57:41,985 WARN [utils.nio.NioConnection] (main:null)
(logid:) Unable to connect to remote: is there a server running on port 8250
2021-05-19 23:57:46,986 INFO [cloud.agent.Agent] (main:null) (logid:)
Connecting to host:79.143.86.149
2021-05-19 23:57:46,987 INFO [utils.nio.NioClient] (ma
Bellow is management server log:
] (Cluster-Heartbeat-1:ctx-f4e82537) (logid:c1947b66) Management node 11 is
detected inactive by timestamp but is pingable
2021-05-20 00:02:52,857 DEBUG [c.c.a.ApiServlet]
(qtp1609124502-279:ctx-17009f08) (logid:7e8b8325) ===START===
2.190.171.255 -- GET
command=prepareHostForMaintenance&id=49bbd35c-9e33-48a5-ba38-41452429986b&response=json&_=1621461469563
2021-05-20 00:02:52,870 DEBUG [c.c.a.ApiServer]
(qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) CIDRs from
which account 'Acct[97f4b3a5-a631-11ea-84fb-002590c1a7d4-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2021-05-20 00:02:52,893 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-8:ctx-4599c29b job-594) (logid:176b8386) Add job-594 into
job monitoring
2021-05-20 00:02:52,897 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) submit async
job-594, details: AsyncJobVO {id:594, userId: 2, accountId: 2,
instanceType: Host, instanceId: 1, cmd:
org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd,
cmdInfo:
{"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"23778","id":"49bbd35c-9e33-48a5-ba38-41452429986b","ctxDetails":"{\"interface
com.cloud.host.Host\":\"49bbd35c-9e33-48a5-ba38-41452429986b\"}","ctxAccountId":"2","uuid":"49bbd35c-9e33-48a5-ba38-41452429986b","cmdEventType":"MAINT.PREPARE","_":"1621461469563"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 279278805467174, completeMsid: null, lastUpdated:
null, lastPolled: null, created: null, removed: null}
2021-05-20 00:02:52,899 DEBUG [c.c.a.ApiServlet]
(qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) ===END===
2.190.171.255 -- GET
command=prepareHostForMaintenance&id=49bbd35c-9e33-48a5-ba38-41452429986b&response=json&_=1621461469563
2021-05-20 00:02:52,899 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Executing
AsyncJobVO {id:594, userId: 2, accountId: 2, instanceType: Host,
instanceId: 1, cmd:
org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd,
cmdInfo:
{"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"23778","id":"49bbd35c-9e33-48a5-ba38-41452429986b","ctxDetails":"{\"interface
com.cloud.host.Host\":\"49bbd35c-9e33-48a5-ba38-41452429986b\"}","ctxAccountId":"2","uuid":"49bbd35c-9e33-48a5-ba38-41452429986b","cmdEventType":"MAINT.PREPARE","_":"1621461469563"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 279278805467174, completeMsid: null, lastUpdated:
null, lastPolled: null, created: null, removed: null}
2021-05-20 00:02:52,924 INFO [c.c.r.ResourceManagerImpl]
(API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646) (logid:33d9ad1e)
Maintenance: attempting maintenance of host
49bbd35c-9e33-48a5-ba38-41452429986b
2021-05-20 00:02:52,926 DEBUG [c.c.a.m.AgentManagerImpl]
(API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646) (logid:33d9ad1e) Can
not send command com.cloud.agent.api.MaintainCommand due to Host 1 is not up
2021-05-20 00:02:52,926 WARN [c.c.r.ResourceManagerImpl]
(API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646) (logid:33d9ad1e)
Unable to send MaintainCommand to host: 1
2021-05-20 00:02:52,928 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Complete async
job-594, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed
to prepare host for maintenance due to: Unable to prepare for maintenance
host 1"}
2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Publish async
job-594 complete on message bus
2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Wake up jobs
related to job-594
2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Update db status
for job-594
2021-05-20 00:02:52,932 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Wake up jobs
joined with job-594 and disjoin all subjobs created from job- 594
2021-05-20 00:02:52,939 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Done executing
org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd for
job-594
2021-05-20 00:02:52,939 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Remove job-594
from job monitoring
2021-05-20 00:02:53,709 DEBUG [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Detected management
node left, id:11, nodeIP:169.254.0.1
2021-05-20 00:02:53,709 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Trying to connect to
169.254.0.1
2021-05-20 00:02:53,709 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Management node 11 is
detected inactive by timestamp but is pingable
2021-05-20 00:02:55,209 DEBUG [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Detected management
node left, id:11, nodeIP:169.254.0.1
2021-05-20 00:02:55,209 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Trying to connect to
169.254.0.1
2021-05-20 00:02:55,210 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Management node 11 is
detected inactive by timestamp but is pingable
2021-05-20 00:02:55,968 DEBUG [c.c.a.ApiServlet]
(qtp1609124502-308:ctx-ef5fd0c0) (logid:ac2a9f7e) ===START===
2.190.171.255 -- GET
command=queryAsyncJobResult&jobId=33d9ad1e-40fb-43db-bfb2-4047d386e9ad&response=json&_=1621461469564
2021-05-20 00:02:55,981 DEBUG [c.c.a.ApiServer]
(qtp1609124502-308:ctx-ef5fd0c0 ctx-1484d35d) (logid:ac2a9f7e) CIDRs from
which account 'Acct[97f4b3a5-a631-11ea-84fb-002590c1a7d4-admin]' is allowed
to perform API calls: 0.0.0.0/0,::/0
2021-05-20 00:02:56,005 DEBUG [c.c.a.ApiServlet]
(qtp1609124502-308:ctx-ef5fd0c0 ctx-1484d35d) (logid:ac2a9f7e) ===END===
2.190.171.255 -- GET
command=queryAsyncJobResult&jobId=33d9ad1e-40fb-43db-bfb2-4047d386e9ad&response=json&_=1621461469564
2021-05-20 00:02:56,708 DEBUG [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Detected management
node left, id:11, nodeIP:169.254.0.1
2021-05-20 00:02:56,708 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Trying to connect to
169.254.0.1
2021-05-20 00:02:56,708 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Management node 11 is
detected inactive by timestamp but is pingable
2021-05-20 00:02:56,948 DEBUG [c.c.s.StatsCollector]
(StatsCollector-3:ctx-a25c34d9) (logid:a111f62a) AutoScaling Monitor is
running...
2021-05-20 00:02:56,971 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-43fd4a09) (logid:2ea9d48e) HostStatsCollector is
running...
2021-05-20 00:02:57,173 DEBUG [c.c.s.StatsCollector]
(StatsCollector-4:ctx-191f8643) (logid:c0c4dc4d) StorageCollector is
running...
2021-05-20 00:02:57,182 DEBUG [c.c.s.StatsCollector]
(StatsCollector-4:ctx-191f8643) (logid:c0c4dc4d) There is no secondary
storage VM for secondary storage host Secondary1
2021-05-20 00:02:58,208 DEBUG [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Detected management
node left, id:11, nodeIP:169.254.0.1
2021-05-20 00:02:58,208 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Trying to connect to
169.254.0.1
2021-05-20 00:02:58,208 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Management node 11 is
detected inactive by timestamp but is pingable
2021-05-20 00:02:59,707 DEBUG [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Detected management
node left, id:11, nodeIP:169.254.0.1
2021-05-20 00:02:59,707 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Trying to connect to
169.254.0.1
2021-05-20 00:02:59,707 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Management node 11 is
detected inactive by timestamp but is pingable
2021-05-20 00:03:01,052 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-5d2e0d48) (logid:1252bec0) Zone 2 is not ready to
launch console proxy yet
2021-05-20 00:03:01,064 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-e131be81) (logid:15af42a6) Zone 2 is not ready to launch
secondary storage VM yet
2021-05-20 00:03:01,207 DEBUG [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Detected management
node left, id:11, nodeIP:169.254.0.1
2021-05-20 00:03:01,207 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Trying to connect to
169.254.0.1
2021-05-20 00:03:01,208 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Management node 11 is
detected inactive by timestamp but is pingable
2021-05-20 00:03:02,707 DEBUG [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Detected management
node left, id:11, nodeIP:169.254.0.1
2021-05-20 00:03:02,708 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Trying to connect to
169.254.0.1
2021-05-20 00:03:02,708 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Management node 11 is
detected inactive by timestamp but is pingable
2021-05-20 00:03:04,206 DEBUG [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Detected management
node left, id:11, nodeIP:169.254.0.1
2021-05-20 00:03:04,206 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Trying to connect to
169.254.0.1
2021-05-20 00:03:04,206 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Management node 11 is
detected inactive by timestamp but is pingable
2021-05-20 00:03:05,708 DEBUG [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Detected management
node left, id:11, nodeIP:169.254.0.1
2021-05-20 00:03:05,708 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Trying to connect to
169.254.0.1
2021-05-20 00:03:05,708 INFO [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Management node 11 is
detected inactive by timestamp but is pingable
[root@ir-kvm1 management]#
Re: Change host IP
Posted by Fariborz Navidan <md...@gmail.com>.
Is it safe to force remove the host?
On Thu, May 20, 2021 at 4:12 PM Fariborz Navidan <md...@gmail.com>
wrote:
> All VMs show as stopped but "virsh list" show dp,aoms rimmomh
>
>
> On Thu, May 20, 2021 at 1:20 PM Yordan Kostov <Yo...@nsogroup.com>
> wrote:
>
>> Hello Fariborz,
>>
>> I am not experienced with ACS + KVM but here is something that
>> may help.
>> If host is empty of VMs, can you force-remove it and add it again?
>>
>> Regards,
>> Jordan
>>
>> -----Original Message-----
>> From: Fariborz Navidan <md...@gmail.com>
>> Sent: Thursday, May 20, 2021 11:23 AM
>> To: users@cloudstack.apache.org
>> Subject: Re: Change host IP
>>
>>
>> [X] This message came from outside your organization
>>
>>
>> When I restart agent the old IP gets back to agent.properties
>>
>> Please help[ me/
>>
>> On Thu, May 20, 2021 at 2:42 AM Fariborz Navidan <md...@gmail.com>
>> wrote:
>>
>> > I use cloudstack 4.14.0
>> >
>> > Please help me
>> >
>> > On Thu, May 20, 2021 at 2:34 AM Fariborz Navidan
>> > <md...@gmail.com>
>> > wrote:
>> >
>> >> Hello,
>> >>
>> >> We have migrated our server and as a result, we need to change the IP
>> >> address. I have modified agent.properties and all ip address fields
>> >> of host record to match the new IP address. But unfortunately,
>> >> management server is unable to connect to it. I forced the host to go
>> >> for maintenance checking "force". Now when I disable maintenance mode
>> >> I get error " Failed to prepare host for maintenance due to: Unable
>> >> to prepare for maintenance host 1".
>> >>
>> >> Bellow is the agent log:
>> >>
>> >> 2021-05-19 23:57:41,985 WARN [utils.nio.NioConnection] (main:null)
>> >> (logid:) Unable to connect to remote: is there a server running on
>> >> port 8250
>> >> 2021-05-19 23:57:46,986 INFO [cloud.agent.Agent] (main:null)
>> >> (logid:) Connecting to host:79.143.86.149
>> >> 2021-05-19 23:57:46,987 INFO [utils.nio.NioClient] (ma
>> >>
>> >> Bellow is management server log:
>> >>
>> >> ] (Cluster-Heartbeat-1:ctx-f4e82537) (logid:c1947b66) Management node
>> >> 11 is detected inactive by timestamp but is pingable
>> >> 2021-05-20 00:02:52,857 DEBUG [c.c.a.ApiServlet]
>> >> (qtp1609124502-279:ctx-17009f08) (logid:7e8b8325) ===START===
>> >> 2.190.171.255 -- GET
>> >>
>> >> command=prepareHostForMaintenance&id=49bbd35c-9e33-48a5-ba38-41452429
>> >> 986b&response=json&_=1621461469563
>> >> 2021-05-20 00:02:52,870 DEBUG [c.c.a.ApiServer]
>> >> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) CIDRs
>> >> from which account 'Acct[97f4b3a5-a631-11ea-84fb-002590c1a7d4-admin]'
>> >> is allowed to perform API calls: 0.0.0.0/0,::/0
>> >> 2021-05-20 00:02:52,893 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:176b8386) Add
>> >> job-594 into job monitoring
>> >> 2021-05-20 00:02:52,897 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) submit
>> >> async job-594, details: AsyncJobVO {id:594, userId: 2, accountId: 2,
>> >> instanceType: Host, instanceId: 1, cmd:
>> >> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd
>> >> ,
>> >> cmdInfo:
>> >> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventI
>> >> d":"23778","id":"49bbd35c-9e33-48a5-ba38-41452429986b","ctxDetails":"
>> >> {\"interface
>> >> com.cloud.host.Host\":\"49bbd35c-9e33-48a5-ba38-41452429986b\"}","ctx
>> >> AccountId":"2","uuid":"49bbd35c-9e33-48a5-ba38-41452429986b","cmdEven
>> >> tType":"MAINT.PREPARE","_":"1621461469563"},
>> >> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> >> result: null, initMsid: 279278805467174, completeMsid: null,
>> lastUpdated:
>> >> null, lastPolled: null, created: null, removed: null}
>> >> 2021-05-20 00:02:52,899 DEBUG [c.c.a.ApiServlet]
>> >> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325)
>> >> ===END===
>> >> 2.190.171.255 -- GET
>> >>
>> >> command=prepareHostForMaintenance&id=49bbd35c-9e33-48a5-ba38-41452429
>> >> 986b&response=json&_=1621461469563
>> >> 2021-05-20 00:02:52,899 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Executing
>> >> AsyncJobVO {id:594, userId: 2, accountId: 2, instanceType: Host,
>> >> instanceId: 1, cmd:
>> >> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd
>> >> ,
>> >> cmdInfo:
>> >> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventI
>> >> d":"23778","id":"49bbd35c-9e33-48a5-ba38-41452429986b","ctxDetails":"
>> >> {\"interface
>> >> com.cloud.host.Host\":\"49bbd35c-9e33-48a5-ba38-41452429986b\"}","ctx
>> >> AccountId":"2","uuid":"49bbd35c-9e33-48a5-ba38-41452429986b","cmdEven
>> >> tType":"MAINT.PREPARE","_":"1621461469563"},
>> >> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> >> result: null, initMsid: 279278805467174, completeMsid: null,
>> lastUpdated:
>> >> null, lastPolled: null, created: null, removed: null}
>> >> 2021-05-20 00:02:52,924 INFO [c.c.r.ResourceManagerImpl]
>> >> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646)
>> >> (logid:33d9ad1e)
>> >> Maintenance: attempting maintenance of host
>> >> 49bbd35c-9e33-48a5-ba38-41452429986b
>> >> 2021-05-20 00:02:52,926 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646)
>> >> (logid:33d9ad1e) Can not send command
>> >> com.cloud.agent.api.MaintainCommand due to Host 1 is not up
>> >> 2021-05-20 00:02:52,926 WARN [c.c.r.ResourceManagerImpl]
>> >> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646)
>> >> (logid:33d9ad1e) Unable to send MaintainCommand to host: 1
>> >> 2021-05-20 00:02:52,928 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Complete
>> >> async job-594, jobStatus: FAILED, resultCode: 530, result:
>> >> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList"
>> >> :[],"errorcode":530,"errortext":"Failed
>> >> to prepare host for maintenance due to: Unable to prepare for
>> >> maintenance host 1"}
>> >> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Publish
>> >> async
>> >> job-594 complete on message bus
>> >> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Wake up
>> >> jobs related to job-594
>> >> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Update db
>> >> status for job-594
>> >> 2021-05-20 00:02:52,932 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Wake up
>> >> jobs joined with job-594 and disjoin all subjobs created from job-
>> >> 594
>> >> 2021-05-20 00:02:52,939 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Done
>> >> executing
>> >> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd
>> >> for
>> >> job-594
>> >> 2021-05-20 00:02:52,939 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Remove
>> >> job-594 from job monitoring
>> >> 2021-05-20 00:02:53,709 DEBUG [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Detected
>> >> management node left, id:11, nodeIP:169.254.0.1
>> >> 2021-05-20 00:02:53,709 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Trying to connect
>> >> to
>> >> 169.254.0.1
>> >> 2021-05-20 00:02:53,709 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Management node
>> >> 11 is detected inactive by timestamp but is pingable
>> >> 2021-05-20 00:02:55,209 DEBUG [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Detected
>> >> management node left, id:11, nodeIP:169.254.0.1
>> >> 2021-05-20 00:02:55,209 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Trying to connect
>> >> to
>> >> 169.254.0.1
>> >> 2021-05-20 00:02:55,210 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Management node
>> >> 11 is detected inactive by timestamp but is pingable
>> >> 2021-05-20 00:02:55,968 DEBUG [c.c.a.ApiServlet]
>> >> (qtp1609124502-308:ctx-ef5fd0c0) (logid:ac2a9f7e) ===START===
>> >> 2.190.171.255 -- GET
>> >>
>> >> command=queryAsyncJobResult&jobId=33d9ad1e-40fb-43db-bfb2-4047d386e9a
>> >> d&response=json&_=1621461469564
>> >> 2021-05-20 00:02:55,981 DEBUG [c.c.a.ApiServer]
>> >> (qtp1609124502-308:ctx-ef5fd0c0 ctx-1484d35d) (logid:ac2a9f7e) CIDRs
>> >> from which account 'Acct[97f4b3a5-a631-11ea-84fb-002590c1a7d4-admin]'
>> >> is allowed to perform API calls: 0.0.0.0/0,::/0
>> >> 2021-05-20 00:02:56,005 DEBUG [c.c.a.ApiServlet]
>> >> (qtp1609124502-308:ctx-ef5fd0c0 ctx-1484d35d) (logid:ac2a9f7e)
>> >> ===END===
>> >> 2.190.171.255 -- GET
>> >>
>> >> command=queryAsyncJobResult&jobId=33d9ad1e-40fb-43db-bfb2-4047d386e9a
>> >> d&response=json&_=1621461469564
>> >> 2021-05-20 00:02:56,708 DEBUG [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Detected
>> >> management node left, id:11, nodeIP:169.254.0.1
>> >> 2021-05-20 00:02:56,708 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Trying to connect
>> >> to
>> >> 169.254.0.1
>> >> 2021-05-20 00:02:56,708 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Management node
>> >> 11 is detected inactive by timestamp but is pingable
>> >> 2021-05-20 00:02:56,948 DEBUG [c.c.s.StatsCollector]
>> >> (StatsCollector-3:ctx-a25c34d9) (logid:a111f62a) AutoScaling Monitor
>> >> is running...
>> >> 2021-05-20 00:02:56,971 DEBUG [c.c.s.StatsCollector]
>> >> (StatsCollector-5:ctx-43fd4a09) (logid:2ea9d48e) HostStatsCollector
>> >> is running...
>> >> 2021-05-20 00:02:57,173 DEBUG [c.c.s.StatsCollector]
>> >> (StatsCollector-4:ctx-191f8643) (logid:c0c4dc4d) StorageCollector is
>> >> running...
>> >> 2021-05-20 00:02:57,182 DEBUG [c.c.s.StatsCollector]
>> >> (StatsCollector-4:ctx-191f8643) (logid:c0c4dc4d) There is no
>> >> secondary storage VM for secondary storage host Secondary1
>> >> 2021-05-20 00:02:58,208 DEBUG [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Detected
>> >> management node left, id:11, nodeIP:169.254.0.1
>> >> 2021-05-20 00:02:58,208 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Trying to connect
>> >> to
>> >> 169.254.0.1
>> >> 2021-05-20 00:02:58,208 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Management node
>> >> 11 is detected inactive by timestamp but is pingable
>> >> 2021-05-20 00:02:59,707 DEBUG [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Detected
>> >> management node left, id:11, nodeIP:169.254.0.1
>> >> 2021-05-20 00:02:59,707 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Trying to connect
>> >> to
>> >> 169.254.0.1
>> >> 2021-05-20 00:02:59,707 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Management node
>> >> 11 is detected inactive by timestamp but is pingable
>> >> 2021-05-20 00:03:01,052 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> >> (consoleproxy-1:ctx-5d2e0d48) (logid:1252bec0) Zone 2 is not ready to
>> >> launch console proxy yet
>> >> 2021-05-20 00:03:01,064 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> >> (secstorage-1:ctx-e131be81) (logid:15af42a6) Zone 2 is not ready to
>> >> launch secondary storage VM yet
>> >> 2021-05-20 00:03:01,207 DEBUG [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Detected
>> >> management node left, id:11, nodeIP:169.254.0.1
>> >> 2021-05-20 00:03:01,207 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Trying to connect
>> >> to
>> >> 169.254.0.1
>> >> 2021-05-20 00:03:01,208 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Management node
>> >> 11 is detected inactive by timestamp but is pingable
>> >> 2021-05-20 00:03:02,707 DEBUG [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Detected
>> >> management node left, id:11, nodeIP:169.254.0.1
>> >> 2021-05-20 00:03:02,708 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Trying to connect
>> >> to
>> >> 169.254.0.1
>> >> 2021-05-20 00:03:02,708 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Management node
>> >> 11 is detected inactive by timestamp but is pingable
>> >> 2021-05-20 00:03:04,206 DEBUG [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Detected
>> >> management node left, id:11, nodeIP:169.254.0.1
>> >> 2021-05-20 00:03:04,206 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Trying to connect
>> >> to
>> >> 169.254.0.1
>> >> 2021-05-20 00:03:04,206 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Management node
>> >> 11 is detected inactive by timestamp but is pingable
>> >> 2021-05-20 00:03:05,708 DEBUG [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Detected
>> >> management node left, id:11, nodeIP:169.254.0.1
>> >> 2021-05-20 00:03:05,708 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Trying to connect
>> >> to
>> >> 169.254.0.1
>> >> 2021-05-20 00:03:05,708 INFO [c.c.c.ClusterManagerImpl]
>> >> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Management node
>> >> 11 is detected inactive by timestamp but is pingable
>> >> [root@ir-kvm1 management]#
>> >>
>> >>
>>
>
Re: Change host IP
Posted by Fariborz Navidan <md...@gmail.com>.
All VMs show as stopped but "virsh list" show dp,aoms rimmomh
On Thu, May 20, 2021 at 1:20 PM Yordan Kostov <Yo...@nsogroup.com> wrote:
> Hello Fariborz,
>
> I am not experienced with ACS + KVM but here is something that may
> help.
> If host is empty of VMs, can you force-remove it and add it again?
>
> Regards,
> Jordan
>
> -----Original Message-----
> From: Fariborz Navidan <md...@gmail.com>
> Sent: Thursday, May 20, 2021 11:23 AM
> To: users@cloudstack.apache.org
> Subject: Re: Change host IP
>
>
> [X] This message came from outside your organization
>
>
> When I restart agent the old IP gets back to agent.properties
>
> Please help[ me/
>
> On Thu, May 20, 2021 at 2:42 AM Fariborz Navidan <md...@gmail.com>
> wrote:
>
> > I use cloudstack 4.14.0
> >
> > Please help me
> >
> > On Thu, May 20, 2021 at 2:34 AM Fariborz Navidan
> > <md...@gmail.com>
> > wrote:
> >
> >> Hello,
> >>
> >> We have migrated our server and as a result, we need to change the IP
> >> address. I have modified agent.properties and all ip address fields
> >> of host record to match the new IP address. But unfortunately,
> >> management server is unable to connect to it. I forced the host to go
> >> for maintenance checking "force". Now when I disable maintenance mode
> >> I get error " Failed to prepare host for maintenance due to: Unable
> >> to prepare for maintenance host 1".
> >>
> >> Bellow is the agent log:
> >>
> >> 2021-05-19 23:57:41,985 WARN [utils.nio.NioConnection] (main:null)
> >> (logid:) Unable to connect to remote: is there a server running on
> >> port 8250
> >> 2021-05-19 23:57:46,986 INFO [cloud.agent.Agent] (main:null)
> >> (logid:) Connecting to host:79.143.86.149
> >> 2021-05-19 23:57:46,987 INFO [utils.nio.NioClient] (ma
> >>
> >> Bellow is management server log:
> >>
> >> ] (Cluster-Heartbeat-1:ctx-f4e82537) (logid:c1947b66) Management node
> >> 11 is detected inactive by timestamp but is pingable
> >> 2021-05-20 00:02:52,857 DEBUG [c.c.a.ApiServlet]
> >> (qtp1609124502-279:ctx-17009f08) (logid:7e8b8325) ===START===
> >> 2.190.171.255 -- GET
> >>
> >> command=prepareHostForMaintenance&id=49bbd35c-9e33-48a5-ba38-41452429
> >> 986b&response=json&_=1621461469563
> >> 2021-05-20 00:02:52,870 DEBUG [c.c.a.ApiServer]
> >> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) CIDRs
> >> from which account 'Acct[97f4b3a5-a631-11ea-84fb-002590c1a7d4-admin]'
> >> is allowed to perform API calls: 0.0.0.0/0,::/0
> >> 2021-05-20 00:02:52,893 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:176b8386) Add
> >> job-594 into job monitoring
> >> 2021-05-20 00:02:52,897 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) submit
> >> async job-594, details: AsyncJobVO {id:594, userId: 2, accountId: 2,
> >> instanceType: Host, instanceId: 1, cmd:
> >> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd
> >> ,
> >> cmdInfo:
> >> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventI
> >> d":"23778","id":"49bbd35c-9e33-48a5-ba38-41452429986b","ctxDetails":"
> >> {\"interface
> >> com.cloud.host.Host\":\"49bbd35c-9e33-48a5-ba38-41452429986b\"}","ctx
> >> AccountId":"2","uuid":"49bbd35c-9e33-48a5-ba38-41452429986b","cmdEven
> >> tType":"MAINT.PREPARE","_":"1621461469563"},
> >> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> >> result: null, initMsid: 279278805467174, completeMsid: null,
> lastUpdated:
> >> null, lastPolled: null, created: null, removed: null}
> >> 2021-05-20 00:02:52,899 DEBUG [c.c.a.ApiServlet]
> >> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325)
> >> ===END===
> >> 2.190.171.255 -- GET
> >>
> >> command=prepareHostForMaintenance&id=49bbd35c-9e33-48a5-ba38-41452429
> >> 986b&response=json&_=1621461469563
> >> 2021-05-20 00:02:52,899 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Executing
> >> AsyncJobVO {id:594, userId: 2, accountId: 2, instanceType: Host,
> >> instanceId: 1, cmd:
> >> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd
> >> ,
> >> cmdInfo:
> >> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventI
> >> d":"23778","id":"49bbd35c-9e33-48a5-ba38-41452429986b","ctxDetails":"
> >> {\"interface
> >> com.cloud.host.Host\":\"49bbd35c-9e33-48a5-ba38-41452429986b\"}","ctx
> >> AccountId":"2","uuid":"49bbd35c-9e33-48a5-ba38-41452429986b","cmdEven
> >> tType":"MAINT.PREPARE","_":"1621461469563"},
> >> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> >> result: null, initMsid: 279278805467174, completeMsid: null,
> lastUpdated:
> >> null, lastPolled: null, created: null, removed: null}
> >> 2021-05-20 00:02:52,924 INFO [c.c.r.ResourceManagerImpl]
> >> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646)
> >> (logid:33d9ad1e)
> >> Maintenance: attempting maintenance of host
> >> 49bbd35c-9e33-48a5-ba38-41452429986b
> >> 2021-05-20 00:02:52,926 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646)
> >> (logid:33d9ad1e) Can not send command
> >> com.cloud.agent.api.MaintainCommand due to Host 1 is not up
> >> 2021-05-20 00:02:52,926 WARN [c.c.r.ResourceManagerImpl]
> >> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646)
> >> (logid:33d9ad1e) Unable to send MaintainCommand to host: 1
> >> 2021-05-20 00:02:52,928 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Complete
> >> async job-594, jobStatus: FAILED, resultCode: 530, result:
> >> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList"
> >> :[],"errorcode":530,"errortext":"Failed
> >> to prepare host for maintenance due to: Unable to prepare for
> >> maintenance host 1"}
> >> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Publish
> >> async
> >> job-594 complete on message bus
> >> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Wake up
> >> jobs related to job-594
> >> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Update db
> >> status for job-594
> >> 2021-05-20 00:02:52,932 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Wake up
> >> jobs joined with job-594 and disjoin all subjobs created from job-
> >> 594
> >> 2021-05-20 00:02:52,939 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Done
> >> executing
> >> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd
> >> for
> >> job-594
> >> 2021-05-20 00:02:52,939 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Remove
> >> job-594 from job monitoring
> >> 2021-05-20 00:02:53,709 DEBUG [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Detected
> >> management node left, id:11, nodeIP:169.254.0.1
> >> 2021-05-20 00:02:53,709 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Trying to connect
> >> to
> >> 169.254.0.1
> >> 2021-05-20 00:02:53,709 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Management node
> >> 11 is detected inactive by timestamp but is pingable
> >> 2021-05-20 00:02:55,209 DEBUG [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Detected
> >> management node left, id:11, nodeIP:169.254.0.1
> >> 2021-05-20 00:02:55,209 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Trying to connect
> >> to
> >> 169.254.0.1
> >> 2021-05-20 00:02:55,210 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Management node
> >> 11 is detected inactive by timestamp but is pingable
> >> 2021-05-20 00:02:55,968 DEBUG [c.c.a.ApiServlet]
> >> (qtp1609124502-308:ctx-ef5fd0c0) (logid:ac2a9f7e) ===START===
> >> 2.190.171.255 -- GET
> >>
> >> command=queryAsyncJobResult&jobId=33d9ad1e-40fb-43db-bfb2-4047d386e9a
> >> d&response=json&_=1621461469564
> >> 2021-05-20 00:02:55,981 DEBUG [c.c.a.ApiServer]
> >> (qtp1609124502-308:ctx-ef5fd0c0 ctx-1484d35d) (logid:ac2a9f7e) CIDRs
> >> from which account 'Acct[97f4b3a5-a631-11ea-84fb-002590c1a7d4-admin]'
> >> is allowed to perform API calls: 0.0.0.0/0,::/0
> >> 2021-05-20 00:02:56,005 DEBUG [c.c.a.ApiServlet]
> >> (qtp1609124502-308:ctx-ef5fd0c0 ctx-1484d35d) (logid:ac2a9f7e)
> >> ===END===
> >> 2.190.171.255 -- GET
> >>
> >> command=queryAsyncJobResult&jobId=33d9ad1e-40fb-43db-bfb2-4047d386e9a
> >> d&response=json&_=1621461469564
> >> 2021-05-20 00:02:56,708 DEBUG [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Detected
> >> management node left, id:11, nodeIP:169.254.0.1
> >> 2021-05-20 00:02:56,708 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Trying to connect
> >> to
> >> 169.254.0.1
> >> 2021-05-20 00:02:56,708 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Management node
> >> 11 is detected inactive by timestamp but is pingable
> >> 2021-05-20 00:02:56,948 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-3:ctx-a25c34d9) (logid:a111f62a) AutoScaling Monitor
> >> is running...
> >> 2021-05-20 00:02:56,971 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-5:ctx-43fd4a09) (logid:2ea9d48e) HostStatsCollector
> >> is running...
> >> 2021-05-20 00:02:57,173 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-4:ctx-191f8643) (logid:c0c4dc4d) StorageCollector is
> >> running...
> >> 2021-05-20 00:02:57,182 DEBUG [c.c.s.StatsCollector]
> >> (StatsCollector-4:ctx-191f8643) (logid:c0c4dc4d) There is no
> >> secondary storage VM for secondary storage host Secondary1
> >> 2021-05-20 00:02:58,208 DEBUG [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Detected
> >> management node left, id:11, nodeIP:169.254.0.1
> >> 2021-05-20 00:02:58,208 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Trying to connect
> >> to
> >> 169.254.0.1
> >> 2021-05-20 00:02:58,208 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Management node
> >> 11 is detected inactive by timestamp but is pingable
> >> 2021-05-20 00:02:59,707 DEBUG [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Detected
> >> management node left, id:11, nodeIP:169.254.0.1
> >> 2021-05-20 00:02:59,707 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Trying to connect
> >> to
> >> 169.254.0.1
> >> 2021-05-20 00:02:59,707 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Management node
> >> 11 is detected inactive by timestamp but is pingable
> >> 2021-05-20 00:03:01,052 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> >> (consoleproxy-1:ctx-5d2e0d48) (logid:1252bec0) Zone 2 is not ready to
> >> launch console proxy yet
> >> 2021-05-20 00:03:01,064 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> >> (secstorage-1:ctx-e131be81) (logid:15af42a6) Zone 2 is not ready to
> >> launch secondary storage VM yet
> >> 2021-05-20 00:03:01,207 DEBUG [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Detected
> >> management node left, id:11, nodeIP:169.254.0.1
> >> 2021-05-20 00:03:01,207 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Trying to connect
> >> to
> >> 169.254.0.1
> >> 2021-05-20 00:03:01,208 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Management node
> >> 11 is detected inactive by timestamp but is pingable
> >> 2021-05-20 00:03:02,707 DEBUG [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Detected
> >> management node left, id:11, nodeIP:169.254.0.1
> >> 2021-05-20 00:03:02,708 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Trying to connect
> >> to
> >> 169.254.0.1
> >> 2021-05-20 00:03:02,708 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Management node
> >> 11 is detected inactive by timestamp but is pingable
> >> 2021-05-20 00:03:04,206 DEBUG [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Detected
> >> management node left, id:11, nodeIP:169.254.0.1
> >> 2021-05-20 00:03:04,206 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Trying to connect
> >> to
> >> 169.254.0.1
> >> 2021-05-20 00:03:04,206 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Management node
> >> 11 is detected inactive by timestamp but is pingable
> >> 2021-05-20 00:03:05,708 DEBUG [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Detected
> >> management node left, id:11, nodeIP:169.254.0.1
> >> 2021-05-20 00:03:05,708 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Trying to connect
> >> to
> >> 169.254.0.1
> >> 2021-05-20 00:03:05,708 INFO [c.c.c.ClusterManagerImpl]
> >> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Management node
> >> 11 is detected inactive by timestamp but is pingable
> >> [root@ir-kvm1 management]#
> >>
> >>
>
RE: Change host IP
Posted by Yordan Kostov <Yo...@NSOGROUP.COM>.
Hello Fariborz,
I am not experienced with ACS + KVM but here is something that may help.
If host is empty of VMs, can you force-remove it and add it again?
Regards,
Jordan
-----Original Message-----
From: Fariborz Navidan <md...@gmail.com>
Sent: Thursday, May 20, 2021 11:23 AM
To: users@cloudstack.apache.org
Subject: Re: Change host IP
[X] This message came from outside your organization
When I restart agent the old IP gets back to agent.properties
Please help[ me/
On Thu, May 20, 2021 at 2:42 AM Fariborz Navidan <md...@gmail.com>
wrote:
> I use cloudstack 4.14.0
>
> Please help me
>
> On Thu, May 20, 2021 at 2:34 AM Fariborz Navidan
> <md...@gmail.com>
> wrote:
>
>> Hello,
>>
>> We have migrated our server and as a result, we need to change the IP
>> address. I have modified agent.properties and all ip address fields
>> of host record to match the new IP address. But unfortunately,
>> management server is unable to connect to it. I forced the host to go
>> for maintenance checking "force". Now when I disable maintenance mode
>> I get error " Failed to prepare host for maintenance due to: Unable
>> to prepare for maintenance host 1".
>>
>> Bellow is the agent log:
>>
>> 2021-05-19 23:57:41,985 WARN [utils.nio.NioConnection] (main:null)
>> (logid:) Unable to connect to remote: is there a server running on
>> port 8250
>> 2021-05-19 23:57:46,986 INFO [cloud.agent.Agent] (main:null)
>> (logid:) Connecting to host:79.143.86.149
>> 2021-05-19 23:57:46,987 INFO [utils.nio.NioClient] (ma
>>
>> Bellow is management server log:
>>
>> ] (Cluster-Heartbeat-1:ctx-f4e82537) (logid:c1947b66) Management node
>> 11 is detected inactive by timestamp but is pingable
>> 2021-05-20 00:02:52,857 DEBUG [c.c.a.ApiServlet]
>> (qtp1609124502-279:ctx-17009f08) (logid:7e8b8325) ===START===
>> 2.190.171.255 -- GET
>>
>> command=prepareHostForMaintenance&id=49bbd35c-9e33-48a5-ba38-41452429
>> 986b&response=json&_=1621461469563
>> 2021-05-20 00:02:52,870 DEBUG [c.c.a.ApiServer]
>> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) CIDRs
>> from which account 'Acct[97f4b3a5-a631-11ea-84fb-002590c1a7d4-admin]'
>> is allowed to perform API calls: 0.0.0.0/0,::/0
>> 2021-05-20 00:02:52,893 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:176b8386) Add
>> job-594 into job monitoring
>> 2021-05-20 00:02:52,897 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) submit
>> async job-594, details: AsyncJobVO {id:594, userId: 2, accountId: 2,
>> instanceType: Host, instanceId: 1, cmd:
>> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd
>> ,
>> cmdInfo:
>> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventI
>> d":"23778","id":"49bbd35c-9e33-48a5-ba38-41452429986b","ctxDetails":"
>> {\"interface
>> com.cloud.host.Host\":\"49bbd35c-9e33-48a5-ba38-41452429986b\"}","ctx
>> AccountId":"2","uuid":"49bbd35c-9e33-48a5-ba38-41452429986b","cmdEven
>> tType":"MAINT.PREPARE","_":"1621461469563"},
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 279278805467174, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: null, removed: null}
>> 2021-05-20 00:02:52,899 DEBUG [c.c.a.ApiServlet]
>> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325)
>> ===END===
>> 2.190.171.255 -- GET
>>
>> command=prepareHostForMaintenance&id=49bbd35c-9e33-48a5-ba38-41452429
>> 986b&response=json&_=1621461469563
>> 2021-05-20 00:02:52,899 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Executing
>> AsyncJobVO {id:594, userId: 2, accountId: 2, instanceType: Host,
>> instanceId: 1, cmd:
>> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd
>> ,
>> cmdInfo:
>> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventI
>> d":"23778","id":"49bbd35c-9e33-48a5-ba38-41452429986b","ctxDetails":"
>> {\"interface
>> com.cloud.host.Host\":\"49bbd35c-9e33-48a5-ba38-41452429986b\"}","ctx
>> AccountId":"2","uuid":"49bbd35c-9e33-48a5-ba38-41452429986b","cmdEven
>> tType":"MAINT.PREPARE","_":"1621461469563"},
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 279278805467174, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: null, removed: null}
>> 2021-05-20 00:02:52,924 INFO [c.c.r.ResourceManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646)
>> (logid:33d9ad1e)
>> Maintenance: attempting maintenance of host
>> 49bbd35c-9e33-48a5-ba38-41452429986b
>> 2021-05-20 00:02:52,926 DEBUG [c.c.a.m.AgentManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646)
>> (logid:33d9ad1e) Can not send command
>> com.cloud.agent.api.MaintainCommand due to Host 1 is not up
>> 2021-05-20 00:02:52,926 WARN [c.c.r.ResourceManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646)
>> (logid:33d9ad1e) Unable to send MaintainCommand to host: 1
>> 2021-05-20 00:02:52,928 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Complete
>> async job-594, jobStatus: FAILED, resultCode: 530, result:
>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList"
>> :[],"errorcode":530,"errortext":"Failed
>> to prepare host for maintenance due to: Unable to prepare for
>> maintenance host 1"}
>> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Publish
>> async
>> job-594 complete on message bus
>> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Wake up
>> jobs related to job-594
>> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Update db
>> status for job-594
>> 2021-05-20 00:02:52,932 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Wake up
>> jobs joined with job-594 and disjoin all subjobs created from job-
>> 594
>> 2021-05-20 00:02:52,939 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Done
>> executing
>> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd
>> for
>> job-594
>> 2021-05-20 00:02:52,939 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Remove
>> job-594 from job monitoring
>> 2021-05-20 00:02:53,709 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Detected
>> management node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:02:53,709 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Trying to connect
>> to
>> 169.254.0.1
>> 2021-05-20 00:02:53,709 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Management node
>> 11 is detected inactive by timestamp but is pingable
>> 2021-05-20 00:02:55,209 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Detected
>> management node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:02:55,209 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Trying to connect
>> to
>> 169.254.0.1
>> 2021-05-20 00:02:55,210 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Management node
>> 11 is detected inactive by timestamp but is pingable
>> 2021-05-20 00:02:55,968 DEBUG [c.c.a.ApiServlet]
>> (qtp1609124502-308:ctx-ef5fd0c0) (logid:ac2a9f7e) ===START===
>> 2.190.171.255 -- GET
>>
>> command=queryAsyncJobResult&jobId=33d9ad1e-40fb-43db-bfb2-4047d386e9a
>> d&response=json&_=1621461469564
>> 2021-05-20 00:02:55,981 DEBUG [c.c.a.ApiServer]
>> (qtp1609124502-308:ctx-ef5fd0c0 ctx-1484d35d) (logid:ac2a9f7e) CIDRs
>> from which account 'Acct[97f4b3a5-a631-11ea-84fb-002590c1a7d4-admin]'
>> is allowed to perform API calls: 0.0.0.0/0,::/0
>> 2021-05-20 00:02:56,005 DEBUG [c.c.a.ApiServlet]
>> (qtp1609124502-308:ctx-ef5fd0c0 ctx-1484d35d) (logid:ac2a9f7e)
>> ===END===
>> 2.190.171.255 -- GET
>>
>> command=queryAsyncJobResult&jobId=33d9ad1e-40fb-43db-bfb2-4047d386e9a
>> d&response=json&_=1621461469564
>> 2021-05-20 00:02:56,708 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Detected
>> management node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:02:56,708 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Trying to connect
>> to
>> 169.254.0.1
>> 2021-05-20 00:02:56,708 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Management node
>> 11 is detected inactive by timestamp but is pingable
>> 2021-05-20 00:02:56,948 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-3:ctx-a25c34d9) (logid:a111f62a) AutoScaling Monitor
>> is running...
>> 2021-05-20 00:02:56,971 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-5:ctx-43fd4a09) (logid:2ea9d48e) HostStatsCollector
>> is running...
>> 2021-05-20 00:02:57,173 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-4:ctx-191f8643) (logid:c0c4dc4d) StorageCollector is
>> running...
>> 2021-05-20 00:02:57,182 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-4:ctx-191f8643) (logid:c0c4dc4d) There is no
>> secondary storage VM for secondary storage host Secondary1
>> 2021-05-20 00:02:58,208 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Detected
>> management node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:02:58,208 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Trying to connect
>> to
>> 169.254.0.1
>> 2021-05-20 00:02:58,208 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Management node
>> 11 is detected inactive by timestamp but is pingable
>> 2021-05-20 00:02:59,707 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Detected
>> management node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:02:59,707 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Trying to connect
>> to
>> 169.254.0.1
>> 2021-05-20 00:02:59,707 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Management node
>> 11 is detected inactive by timestamp but is pingable
>> 2021-05-20 00:03:01,052 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-5d2e0d48) (logid:1252bec0) Zone 2 is not ready to
>> launch console proxy yet
>> 2021-05-20 00:03:01,064 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-e131be81) (logid:15af42a6) Zone 2 is not ready to
>> launch secondary storage VM yet
>> 2021-05-20 00:03:01,207 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Detected
>> management node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:03:01,207 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Trying to connect
>> to
>> 169.254.0.1
>> 2021-05-20 00:03:01,208 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Management node
>> 11 is detected inactive by timestamp but is pingable
>> 2021-05-20 00:03:02,707 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Detected
>> management node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:03:02,708 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Trying to connect
>> to
>> 169.254.0.1
>> 2021-05-20 00:03:02,708 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Management node
>> 11 is detected inactive by timestamp but is pingable
>> 2021-05-20 00:03:04,206 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Detected
>> management node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:03:04,206 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Trying to connect
>> to
>> 169.254.0.1
>> 2021-05-20 00:03:04,206 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Management node
>> 11 is detected inactive by timestamp but is pingable
>> 2021-05-20 00:03:05,708 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Detected
>> management node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:03:05,708 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Trying to connect
>> to
>> 169.254.0.1
>> 2021-05-20 00:03:05,708 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Management node
>> 11 is detected inactive by timestamp but is pingable
>> [root@ir-kvm1 management]#
>>
>>
Re: Change host IP
Posted by Fariborz Navidan <md...@gmail.com>.
When I restart agent the old IP gets back to agent.properties
Please help[ me/
On Thu, May 20, 2021 at 2:42 AM Fariborz Navidan <md...@gmail.com>
wrote:
> I use cloudstack 4.14.0
>
> Please help me
>
> On Thu, May 20, 2021 at 2:34 AM Fariborz Navidan <md...@gmail.com>
> wrote:
>
>> Hello,
>>
>> We have migrated our server and as a result, we need to change the IP
>> address. I have modified agent.properties and all ip address fields of host
>> record to match the new IP address. But unfortunately, management server is
>> unable to connect to it. I forced the host to go for maintenance checking
>> "force". Now when I disable maintenance mode I get error " Failed to
>> prepare host for maintenance due to: Unable to prepare for maintenance host
>> 1".
>>
>> Bellow is the agent log:
>>
>> 2021-05-19 23:57:41,985 WARN [utils.nio.NioConnection] (main:null)
>> (logid:) Unable to connect to remote: is there a server running on port 8250
>> 2021-05-19 23:57:46,986 INFO [cloud.agent.Agent] (main:null) (logid:)
>> Connecting to host:79.143.86.149
>> 2021-05-19 23:57:46,987 INFO [utils.nio.NioClient] (ma
>>
>> Bellow is management server log:
>>
>> ] (Cluster-Heartbeat-1:ctx-f4e82537) (logid:c1947b66) Management node 11
>> is detected inactive by timestamp but is pingable
>> 2021-05-20 00:02:52,857 DEBUG [c.c.a.ApiServlet]
>> (qtp1609124502-279:ctx-17009f08) (logid:7e8b8325) ===START===
>> 2.190.171.255 -- GET
>> command=prepareHostForMaintenance&id=49bbd35c-9e33-48a5-ba38-41452429986b&response=json&_=1621461469563
>> 2021-05-20 00:02:52,870 DEBUG [c.c.a.ApiServer]
>> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) CIDRs from
>> which account 'Acct[97f4b3a5-a631-11ea-84fb-002590c1a7d4-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2021-05-20 00:02:52,893 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:176b8386) Add job-594 into
>> job monitoring
>> 2021-05-20 00:02:52,897 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) submit async
>> job-594, details: AsyncJobVO {id:594, userId: 2, accountId: 2,
>> instanceType: Host, instanceId: 1, cmd:
>> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd,
>> cmdInfo:
>> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"23778","id":"49bbd35c-9e33-48a5-ba38-41452429986b","ctxDetails":"{\"interface
>> com.cloud.host.Host\":\"49bbd35c-9e33-48a5-ba38-41452429986b\"}","ctxAccountId":"2","uuid":"49bbd35c-9e33-48a5-ba38-41452429986b","cmdEventType":"MAINT.PREPARE","_":"1621461469563"},
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 279278805467174, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: null, removed: null}
>> 2021-05-20 00:02:52,899 DEBUG [c.c.a.ApiServlet]
>> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) ===END===
>> 2.190.171.255 -- GET
>> command=prepareHostForMaintenance&id=49bbd35c-9e33-48a5-ba38-41452429986b&response=json&_=1621461469563
>> 2021-05-20 00:02:52,899 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Executing
>> AsyncJobVO {id:594, userId: 2, accountId: 2, instanceType: Host,
>> instanceId: 1, cmd:
>> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd,
>> cmdInfo:
>> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"23778","id":"49bbd35c-9e33-48a5-ba38-41452429986b","ctxDetails":"{\"interface
>> com.cloud.host.Host\":\"49bbd35c-9e33-48a5-ba38-41452429986b\"}","ctxAccountId":"2","uuid":"49bbd35c-9e33-48a5-ba38-41452429986b","cmdEventType":"MAINT.PREPARE","_":"1621461469563"},
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 279278805467174, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: null, removed: null}
>> 2021-05-20 00:02:52,924 INFO [c.c.r.ResourceManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646) (logid:33d9ad1e)
>> Maintenance: attempting maintenance of host
>> 49bbd35c-9e33-48a5-ba38-41452429986b
>> 2021-05-20 00:02:52,926 DEBUG [c.c.a.m.AgentManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646) (logid:33d9ad1e) Can
>> not send command com.cloud.agent.api.MaintainCommand due to Host 1 is not up
>> 2021-05-20 00:02:52,926 WARN [c.c.r.ResourceManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646) (logid:33d9ad1e)
>> Unable to send MaintainCommand to host: 1
>> 2021-05-20 00:02:52,928 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Complete async
>> job-594, jobStatus: FAILED, resultCode: 530, result:
>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed
>> to prepare host for maintenance due to: Unable to prepare for maintenance
>> host 1"}
>> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Publish async
>> job-594 complete on message bus
>> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Wake up jobs
>> related to job-594
>> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Update db status
>> for job-594
>> 2021-05-20 00:02:52,932 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Wake up jobs
>> joined with job-594 and disjoin all subjobs created from job- 594
>> 2021-05-20 00:02:52,939 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Done executing
>> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd for
>> job-594
>> 2021-05-20 00:02:52,939 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Remove job-594
>> from job monitoring
>> 2021-05-20 00:02:53,709 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Detected management
>> node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:02:53,709 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Trying to connect to
>> 169.254.0.1
>> 2021-05-20 00:02:53,709 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Management node 11 is
>> detected inactive by timestamp but is pingable
>> 2021-05-20 00:02:55,209 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Detected management
>> node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:02:55,209 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Trying to connect to
>> 169.254.0.1
>> 2021-05-20 00:02:55,210 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Management node 11 is
>> detected inactive by timestamp but is pingable
>> 2021-05-20 00:02:55,968 DEBUG [c.c.a.ApiServlet]
>> (qtp1609124502-308:ctx-ef5fd0c0) (logid:ac2a9f7e) ===START===
>> 2.190.171.255 -- GET
>> command=queryAsyncJobResult&jobId=33d9ad1e-40fb-43db-bfb2-4047d386e9ad&response=json&_=1621461469564
>> 2021-05-20 00:02:55,981 DEBUG [c.c.a.ApiServer]
>> (qtp1609124502-308:ctx-ef5fd0c0 ctx-1484d35d) (logid:ac2a9f7e) CIDRs from
>> which account 'Acct[97f4b3a5-a631-11ea-84fb-002590c1a7d4-admin]' is allowed
>> to perform API calls: 0.0.0.0/0,::/0
>> 2021-05-20 00:02:56,005 DEBUG [c.c.a.ApiServlet]
>> (qtp1609124502-308:ctx-ef5fd0c0 ctx-1484d35d) (logid:ac2a9f7e) ===END===
>> 2.190.171.255 -- GET
>> command=queryAsyncJobResult&jobId=33d9ad1e-40fb-43db-bfb2-4047d386e9ad&response=json&_=1621461469564
>> 2021-05-20 00:02:56,708 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Detected management
>> node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:02:56,708 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Trying to connect to
>> 169.254.0.1
>> 2021-05-20 00:02:56,708 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Management node 11 is
>> detected inactive by timestamp but is pingable
>> 2021-05-20 00:02:56,948 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-3:ctx-a25c34d9) (logid:a111f62a) AutoScaling Monitor is
>> running...
>> 2021-05-20 00:02:56,971 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-5:ctx-43fd4a09) (logid:2ea9d48e) HostStatsCollector is
>> running...
>> 2021-05-20 00:02:57,173 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-4:ctx-191f8643) (logid:c0c4dc4d) StorageCollector is
>> running...
>> 2021-05-20 00:02:57,182 DEBUG [c.c.s.StatsCollector]
>> (StatsCollector-4:ctx-191f8643) (logid:c0c4dc4d) There is no secondary
>> storage VM for secondary storage host Secondary1
>> 2021-05-20 00:02:58,208 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Detected management
>> node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:02:58,208 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Trying to connect to
>> 169.254.0.1
>> 2021-05-20 00:02:58,208 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Management node 11 is
>> detected inactive by timestamp but is pingable
>> 2021-05-20 00:02:59,707 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Detected management
>> node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:02:59,707 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Trying to connect to
>> 169.254.0.1
>> 2021-05-20 00:02:59,707 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Management node 11 is
>> detected inactive by timestamp but is pingable
>> 2021-05-20 00:03:01,052 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-5d2e0d48) (logid:1252bec0) Zone 2 is not ready to
>> launch console proxy yet
>> 2021-05-20 00:03:01,064 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-e131be81) (logid:15af42a6) Zone 2 is not ready to launch
>> secondary storage VM yet
>> 2021-05-20 00:03:01,207 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Detected management
>> node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:03:01,207 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Trying to connect to
>> 169.254.0.1
>> 2021-05-20 00:03:01,208 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Management node 11 is
>> detected inactive by timestamp but is pingable
>> 2021-05-20 00:03:02,707 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Detected management
>> node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:03:02,708 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Trying to connect to
>> 169.254.0.1
>> 2021-05-20 00:03:02,708 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Management node 11 is
>> detected inactive by timestamp but is pingable
>> 2021-05-20 00:03:04,206 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Detected management
>> node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:03:04,206 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Trying to connect to
>> 169.254.0.1
>> 2021-05-20 00:03:04,206 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Management node 11 is
>> detected inactive by timestamp but is pingable
>> 2021-05-20 00:03:05,708 DEBUG [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Detected management
>> node left, id:11, nodeIP:169.254.0.1
>> 2021-05-20 00:03:05,708 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Trying to connect to
>> 169.254.0.1
>> 2021-05-20 00:03:05,708 INFO [c.c.c.ClusterManagerImpl]
>> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Management node 11 is
>> detected inactive by timestamp but is pingable
>> [root@ir-kvm1 management]#
>>
>>
Re: Change host IP
Posted by Fariborz Navidan <md...@gmail.com>.
I use cloudstack 4.14.0
Please help me
On Thu, May 20, 2021 at 2:34 AM Fariborz Navidan <md...@gmail.com>
wrote:
> Hello,
>
> We have migrated our server and as a result, we need to change the IP
> address. I have modified agent.properties and all ip address fields of host
> record to match the new IP address. But unfortunately, management server is
> unable to connect to it. I forced the host to go for maintenance checking
> "force". Now when I disable maintenance mode I get error " Failed to
> prepare host for maintenance due to: Unable to prepare for maintenance host
> 1".
>
> Bellow is the agent log:
>
> 2021-05-19 23:57:41,985 WARN [utils.nio.NioConnection] (main:null)
> (logid:) Unable to connect to remote: is there a server running on port 8250
> 2021-05-19 23:57:46,986 INFO [cloud.agent.Agent] (main:null) (logid:)
> Connecting to host:79.143.86.149
> 2021-05-19 23:57:46,987 INFO [utils.nio.NioClient] (ma
>
> Bellow is management server log:
>
> ] (Cluster-Heartbeat-1:ctx-f4e82537) (logid:c1947b66) Management node 11
> is detected inactive by timestamp but is pingable
> 2021-05-20 00:02:52,857 DEBUG [c.c.a.ApiServlet]
> (qtp1609124502-279:ctx-17009f08) (logid:7e8b8325) ===START===
> 2.190.171.255 -- GET
> command=prepareHostForMaintenance&id=49bbd35c-9e33-48a5-ba38-41452429986b&response=json&_=1621461469563
> 2021-05-20 00:02:52,870 DEBUG [c.c.a.ApiServer]
> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) CIDRs from
> which account 'Acct[97f4b3a5-a631-11ea-84fb-002590c1a7d4-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2021-05-20 00:02:52,893 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:176b8386) Add job-594 into
> job monitoring
> 2021-05-20 00:02:52,897 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) submit async
> job-594, details: AsyncJobVO {id:594, userId: 2, accountId: 2,
> instanceType: Host, instanceId: 1, cmd:
> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd,
> cmdInfo:
> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"23778","id":"49bbd35c-9e33-48a5-ba38-41452429986b","ctxDetails":"{\"interface
> com.cloud.host.Host\":\"49bbd35c-9e33-48a5-ba38-41452429986b\"}","ctxAccountId":"2","uuid":"49bbd35c-9e33-48a5-ba38-41452429986b","cmdEventType":"MAINT.PREPARE","_":"1621461469563"},
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 279278805467174, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: null, removed: null}
> 2021-05-20 00:02:52,899 DEBUG [c.c.a.ApiServlet]
> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) ===END===
> 2.190.171.255 -- GET
> command=prepareHostForMaintenance&id=49bbd35c-9e33-48a5-ba38-41452429986b&response=json&_=1621461469563
> 2021-05-20 00:02:52,899 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Executing
> AsyncJobVO {id:594, userId: 2, accountId: 2, instanceType: Host,
> instanceId: 1, cmd:
> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd,
> cmdInfo:
> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"23778","id":"49bbd35c-9e33-48a5-ba38-41452429986b","ctxDetails":"{\"interface
> com.cloud.host.Host\":\"49bbd35c-9e33-48a5-ba38-41452429986b\"}","ctxAccountId":"2","uuid":"49bbd35c-9e33-48a5-ba38-41452429986b","cmdEventType":"MAINT.PREPARE","_":"1621461469563"},
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> result: null, initMsid: 279278805467174, completeMsid: null, lastUpdated:
> null, lastPolled: null, created: null, removed: null}
> 2021-05-20 00:02:52,924 INFO [c.c.r.ResourceManagerImpl]
> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646) (logid:33d9ad1e)
> Maintenance: attempting maintenance of host
> 49bbd35c-9e33-48a5-ba38-41452429986b
> 2021-05-20 00:02:52,926 DEBUG [c.c.a.m.AgentManagerImpl]
> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646) (logid:33d9ad1e) Can
> not send command com.cloud.agent.api.MaintainCommand due to Host 1 is not up
> 2021-05-20 00:02:52,926 WARN [c.c.r.ResourceManagerImpl]
> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646) (logid:33d9ad1e)
> Unable to send MaintainCommand to host: 1
> 2021-05-20 00:02:52,928 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Complete async
> job-594, jobStatus: FAILED, resultCode: 530, result:
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed
> to prepare host for maintenance due to: Unable to prepare for maintenance
> host 1"}
> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Publish async
> job-594 complete on message bus
> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Wake up jobs
> related to job-594
> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Update db status
> for job-594
> 2021-05-20 00:02:52,932 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Wake up jobs
> joined with job-594 and disjoin all subjobs created from job- 594
> 2021-05-20 00:02:52,939 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Done executing
> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd for
> job-594
> 2021-05-20 00:02:52,939 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Remove job-594
> from job monitoring
> 2021-05-20 00:02:53,709 DEBUG [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Detected management
> node left, id:11, nodeIP:169.254.0.1
> 2021-05-20 00:02:53,709 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Trying to connect to
> 169.254.0.1
> 2021-05-20 00:02:53,709 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Management node 11 is
> detected inactive by timestamp but is pingable
> 2021-05-20 00:02:55,209 DEBUG [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Detected management
> node left, id:11, nodeIP:169.254.0.1
> 2021-05-20 00:02:55,209 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Trying to connect to
> 169.254.0.1
> 2021-05-20 00:02:55,210 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Management node 11 is
> detected inactive by timestamp but is pingable
> 2021-05-20 00:02:55,968 DEBUG [c.c.a.ApiServlet]
> (qtp1609124502-308:ctx-ef5fd0c0) (logid:ac2a9f7e) ===START===
> 2.190.171.255 -- GET
> command=queryAsyncJobResult&jobId=33d9ad1e-40fb-43db-bfb2-4047d386e9ad&response=json&_=1621461469564
> 2021-05-20 00:02:55,981 DEBUG [c.c.a.ApiServer]
> (qtp1609124502-308:ctx-ef5fd0c0 ctx-1484d35d) (logid:ac2a9f7e) CIDRs from
> which account 'Acct[97f4b3a5-a631-11ea-84fb-002590c1a7d4-admin]' is allowed
> to perform API calls: 0.0.0.0/0,::/0
> 2021-05-20 00:02:56,005 DEBUG [c.c.a.ApiServlet]
> (qtp1609124502-308:ctx-ef5fd0c0 ctx-1484d35d) (logid:ac2a9f7e) ===END===
> 2.190.171.255 -- GET
> command=queryAsyncJobResult&jobId=33d9ad1e-40fb-43db-bfb2-4047d386e9ad&response=json&_=1621461469564
> 2021-05-20 00:02:56,708 DEBUG [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Detected management
> node left, id:11, nodeIP:169.254.0.1
> 2021-05-20 00:02:56,708 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Trying to connect to
> 169.254.0.1
> 2021-05-20 00:02:56,708 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Management node 11 is
> detected inactive by timestamp but is pingable
> 2021-05-20 00:02:56,948 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-a25c34d9) (logid:a111f62a) AutoScaling Monitor is
> running...
> 2021-05-20 00:02:56,971 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-5:ctx-43fd4a09) (logid:2ea9d48e) HostStatsCollector is
> running...
> 2021-05-20 00:02:57,173 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-191f8643) (logid:c0c4dc4d) StorageCollector is
> running...
> 2021-05-20 00:02:57,182 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-191f8643) (logid:c0c4dc4d) There is no secondary
> storage VM for secondary storage host Secondary1
> 2021-05-20 00:02:58,208 DEBUG [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Detected management
> node left, id:11, nodeIP:169.254.0.1
> 2021-05-20 00:02:58,208 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Trying to connect to
> 169.254.0.1
> 2021-05-20 00:02:58,208 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Management node 11 is
> detected inactive by timestamp but is pingable
> 2021-05-20 00:02:59,707 DEBUG [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Detected management
> node left, id:11, nodeIP:169.254.0.1
> 2021-05-20 00:02:59,707 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Trying to connect to
> 169.254.0.1
> 2021-05-20 00:02:59,707 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Management node 11 is
> detected inactive by timestamp but is pingable
> 2021-05-20 00:03:01,052 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-5d2e0d48) (logid:1252bec0) Zone 2 is not ready to
> launch console proxy yet
> 2021-05-20 00:03:01,064 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-e131be81) (logid:15af42a6) Zone 2 is not ready to launch
> secondary storage VM yet
> 2021-05-20 00:03:01,207 DEBUG [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Detected management
> node left, id:11, nodeIP:169.254.0.1
> 2021-05-20 00:03:01,207 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Trying to connect to
> 169.254.0.1
> 2021-05-20 00:03:01,208 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Management node 11 is
> detected inactive by timestamp but is pingable
> 2021-05-20 00:03:02,707 DEBUG [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Detected management
> node left, id:11, nodeIP:169.254.0.1
> 2021-05-20 00:03:02,708 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Trying to connect to
> 169.254.0.1
> 2021-05-20 00:03:02,708 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Management node 11 is
> detected inactive by timestamp but is pingable
> 2021-05-20 00:03:04,206 DEBUG [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Detected management
> node left, id:11, nodeIP:169.254.0.1
> 2021-05-20 00:03:04,206 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Trying to connect to
> 169.254.0.1
> 2021-05-20 00:03:04,206 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Management node 11 is
> detected inactive by timestamp but is pingable
> 2021-05-20 00:03:05,708 DEBUG [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Detected management
> node left, id:11, nodeIP:169.254.0.1
> 2021-05-20 00:03:05,708 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Trying to connect to
> 169.254.0.1
> 2021-05-20 00:03:05,708 INFO [c.c.c.ClusterManagerImpl]
> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Management node 11 is
> detected inactive by timestamp but is pingable
> [root@ir-kvm1 management]#
>
>