You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Peter Lee <le...@gmail.com> on 2021/06/14 22:08:05 UTC
Fwd: Kafkabroker log swap writing error
---------- Forwarded message ---------
보낸사람: hello <le...@gmail.com>
Date: 2021년 6월 9일 (수) 오후 2:50
Subject: Kafkabroker log swap writing error
To: users-sc.1623217635.lobeokoidbfekkpndnbk-leepeter2019=
gmail.com@kafka.apache.org
<users-sc.1623217635.lobeokoidbfekkpndnbk-leepeter2019=
gmail.com@kafka.apache.org>
Whenever I run Kafka server in windows 10, it failed to rename
controller.log to controller.log.time
After several failed attempts, it stops running.
*java.nio.file.FileSystemException:
C:\Testing\.logskafka-logs\__consumer_offsets-31\00000000000000000000.index.cleaned
->
C:\Testing\.logskafka-logs\__consumer_offsets-31\00000000000000000000.index.swap:*
*“The process cannot access, because the other process is in use”*
*[2021-04-21 10:27:59,232] ERROR Shutdown broker because all log dirs in
C:\Testing\.logskafka-logs have failed (kafka.log.LogManager*
Do you have any idea how to fix this?
[2021-04-20 15:47:53,282] INFO [GroupCoordinator 0]: Stabilized group
monitoragent generation 9 (__consumer_offsets-31)
(kafka.coordinator.group.GroupCoordinator)
[2021-04-20 15:47:59,474] INFO [GroupCoordinator 0]: Preparing to rebalance
group monitoragent in state PreparingRebalance with old generation 9
(__consumer_offsets-31) (reason: Adding new member
node1-agent8-16cc2029-7ee0-4560-9890-028fea05b8b2 with group instance id
None) (kafka.coordinator.group.GroupCoordinator)
[2021-04-20 15:52:53,286] INFO [GroupCoordinator 0]: Member
node1-agent8-4e7ee271-581a-40b2-9b1f-e5fb4f364704 in group monitoragent has
failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2021-04-20 15:52:53,287] INFO [GroupCoordinator 0]: Stabilized group
monitoragent generation 10 (__consumer_offsets-31)
(kafka.coordinator.group.GroupCoordinator)
[2021-04-20 15:52:53,295] INFO [GroupCoordinator 0]: Assignment received
from leader for group monitoragent for generation 10
(kafka.coordinator.group.GroupCoordinator)
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-15].
[2021-04-20 16:06:59,437] INFO [GroupCoordinator 0]: Member
node1-agent8-16cc2029-7ee0-4560-9890-028fea05b8b2 in group monitoragent has
failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/server.log] to
[C:\kafka_2.13-2.7.0/logs/server.log.2021-04-20-15].
[2021-04-20 16:06:59,438] INFO [GroupCoordinator 0]: Preparing to rebalance
group monitoragent in state PreparingRebalance with old generation 10
(__consumer_offsets-31) (reason: removing member
node1-agent8-16cc2029-7ee0-4560-9890-028fea05b8b2 on heartbeat expiration)
(kafka.coordinator.group.GroupCoordinator)
[2021-04-20 16:06:59,439] INFO [GroupCoordinator 0]: Group monitoragent
with generation 11 is now empty (__consumer_offsets-31)
(kafka.coordinator.group.GroupCoordinator)
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-16].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-17].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-18].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-19].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-20].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-21].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-22].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-23].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-00].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-01].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-02].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-03].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-04].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-05].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-06].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-07].
[2021-04-21 08:11:56,092] INFO [GroupCoordinator 0]: Preparing to rebalance
group monitoragent in state PreparingRebalance with old generation 11
(__consumer_offsets-31) (reason: Adding new member
node1-agent8-bd039ca4-dca7-43a9-bd31-48b6a9d86031 with group instance id
None) (kafka.coordinator.group.GroupCoordinator)
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/server.log] to
[C:\kafka_2.13-2.7.0/logs/server.log.2021-04-20-16].
[2021-04-21 08:11:56,095] INFO [GroupCoordinator 0]: Stabilized group
monitoragent generation 12 (__consumer_offsets-31)
(kafka.coordinator.group.GroupCoordinator)
[2021-04-21 08:11:56,100] INFO [GroupCoordinator 0]: Assignment received
from leader for group monitoragent for generation 12
(kafka.coordinator.group.GroupCoordinator)
[2021-04-21 08:16:46,832] INFO [GroupCoordinator 0]: Preparing to rebalance
group monitoragent in state PreparingRebalance with old generation 12
(__consumer_offsets-31) (reason: Adding new member
node1-agent8-4a3f8f6f-1091-4954-99c1-d1ff2c8ae12e with group instance id
None) (kafka.coordinator.group.GroupCoordinator)
[2021-04-21 08:20:22,323] INFO [GroupCoordinator 0]: Member
node1-agent8-bd039ca4-dca7-43a9-bd31-48b6a9d86031 in group monitoragent has
failed, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
[2021-04-21 08:20:22,326] INFO [GroupCoordinator 0]: Stabilized group
monitoragent generation 13 (__consumer_offsets-31)
(kafka.coordinator.group.GroupCoordinator)
[2021-04-21 08:20:22,383] INFO [GroupCoordinator 0]: Assignment received
from leader for group monitoragent for generation 13
(kafka.coordinator.group.GroupCoordinator)
[2021-04-21 08:23:38,927] INFO [GroupCoordinator 0]: Preparing to rebalance
group monitoragent in state PreparingRebalance with old generation 13
(__consumer_offsets-31) (reason: Adding new member
node2-agent9-2210cbbe-9324-4d53-9b0e-a37997275127 with group instance id
None) (kafka.coordinator.group.GroupCoordinator)
[2021-04-21 08:26:23,712] INFO [GroupCoordinator 0]: Stabilized group
monitoragent generation 14 (__consumer_offsets-31)
(kafka.coordinator.group.GroupCoordinator)
[2021-04-21 08:26:23,722] INFO [GroupCoordinator 0]: Assignment received
from leader for group monitoragent for generation 14
(kafka.coordinator.group.GroupCoordinator)
[2021-04-21 09:01:02,878] INFO [GroupCoordinator 0]: Member[
group.instance.id None, member.id
node1-agent8-4a3f8f6f-1091-4954-99c1-d1ff2c8ae12e]
in group monitoragent has left, removing it from the group
(kafka.coordinator.group.GroupCoordinator)
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/server.log] to
[C:\kafka_2.13-2.7.0/logs/server.log.2021-04-21-08].
[2021-04-21 09:01:02,880] INFO [GroupCoordinator 0]: Preparing to rebalance
group monitoragent in state PreparingRebalance with old generation 14
(__consumer_offsets-31) (reason: removing member
node1-agent8-4a3f8f6f-1091-4954-99c1-d1ff2c8ae12e on LeaveGroup)
(kafka.coordinator.group.GroupCoordinator)
[2021-04-21 09:02:27,729] INFO [GroupCoordinator 0]: Stabilized group
monitoragent generation 15 (__consumer_offsets-31)
(kafka.coordinator.group.GroupCoordinator)
[2021-04-21 09:02:27,786] INFO [GroupCoordinator 0]: Assignment received
from leader for group monitoragent for generation 15
(kafka.coordinator.group.GroupCoordinator)
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-08].
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
[C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-09].
[2021-04-21 10:27:53,405] INFO [ProducerStateManager
partition=__consumer_offsets-31] Writing producer snapshot at offset 70374
(kafka.log.ProducerStateManager)
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/server.log] to
[C:\kafka_2.13-2.7.0/logs/server.log.2021-04-21-09].
[2021-04-21 10:27:53,412] INFO [Log partition=__consumer_offsets-31,
dir=C:\Testing\.logskafka-logs] Rolled new log segment at offset 70374 in
11 ms. (kafka.log.Log)
log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/log-cleaner.log] to
[C:\kafka_2.13-2.7.0/logs/log-cleaner.log.2021-04-20-15].
[2021-04-21 10:27:59,102] ERROR Failed to clean up log for
__consumer_offsets-31 in dir C:\Testing\.logskafka-logs due to IOException
(kafka.server.LogDirFailureChannel)
java.nio.file.FileSystemException:
C:\Testing\.logskafka-logs\__consumer_offsets-31\00000000000000000000.index.cleaned
->
C:\Testing\.logskafka-logs\__consumer_offsets-31\00000000000000000000.index.swap:
“The process cannot access, because the other process is in use”
at sun.nio.fs.WindowsException.translateToIOException(Unknown
Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source)
at java.nio.file.Files.move(Unknown Source)
at
org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:917)
at kafka.log.AbstractIndex.renameTo(AbstractIndex.scala:211)
at kafka.log.LazyIndex$IndexValue.renameTo(LazyIndex.scala:155)
at kafka.log.LazyIndex.$anonfun$renameTo$1(LazyIndex.scala:79)
at kafka.log.LazyIndex.renameTo(LazyIndex.scala:79)
at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:496)
at kafka.log.Log.$anonfun$replaceSegments$4(Log.scala:2338)
at kafka.log.Log.$anonfun$replaceSegments$4$adapted(Log.scala:2338)
at scala.collection.immutable.List.foreach(List.scala:333)
at kafka.log.Log.replaceSegments(Log.scala:2338)
at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:614)
at kafka.log.Cleaner.$anonfun$doClean$6(LogCleaner.scala:539)
at kafka.log.Cleaner.doClean(LogCleaner.scala:538)
nsumer_offsets-31\00000000000000000000.index.swap: “The process cannot
access, because the other process is in use”
at
sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown
Source)
at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source)
at java.nio.file.Files.move(Unknown Source)
at
org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:914)
... 18 more
[2021-04-21 10:27:59,120] WARN [ReplicaManager broker=0] Stopping serving
replicas in dir C:\Testing\.logskafka-logs (kafka.server.ReplicaManager)
[2021-04-21 10:27:59,124] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions HashSet(fuzz_seed-36, __consumer_offsets-35,
__consumer_offsets-37, fuzz_seed-42, fuzz_seed-34, fuzz_seed-1,
__consumer_offsets-22, fuzz_seed-17, __consumer_offsets-30, fuzz_seed-12,
fuzz_seed-47, fuzz_seed-27, __consumer_offsets-8, __consumer_offsets-21,
__consumer_offsets-4, __consumer_offsets-27, __consumer_offsets-7,
__consumer_offsets-9, __consumer_offsets-46, fuzz_seed-33, fuzz_seed-2,
fuzz_seed-18, fuzz_seed-0, fuzz_seed-43, __consumer_offsets-25,
__consumer_offsets-41, __consumer_offsets-23, __consumer_offsets-49,
fuzz_seed-26, fuzz_seed-19, __consumer_offsets-33, fuzz_seed-30,
fuzz_seed-39, fuzz_seed-14, __consumer_offsets-47, __consumer_offsets-16,
fuzz_seed-3, fuzz_seed-48, __consumer_offsets-28, fuzz_seed-13,
__consumer_offsets-31, __consumer_offsets-36, __consumer_offsets-42,
__consumer_offsets-3, __consumer_offsets-18, fuzz_seed-46, fuzz_seed-31,
fuzz_seed-9, fuzz_seed-35, __consumer_offsets-15, __consumer_offsets-24,
fuzz_seed-29, fuzz_seed-4, fuzz_seed-20, fuzz_seed-23,
__consumer_offsets-38, __consumer_offsets-48, fuzz_seed-45,
__consumer_offsets-17, fuzz_seed-49, fuzz_seed-40, __consumer_offsets-19,
__consumer_offsets-11, fuzz_seed-21, fuzz_seed-16, fuzz_seed-8,
__consumer_offsets-13, fuzz_seed-10, __consumer_offsets-2,
__consumer_offsets-43, __consumer_offsets-6, fuzz_seed-38,
__consumer_offsets-14, fuzz_seed-25, fuzz_seed-22, fuzz_seed-6,
fuzz_seed-41, __consumer_offsets-20, __consumer_offsets-0, fuzz_seed-37,
fuzz_seed-32, fuzz_seed-11, __consumer_offsets-44, fuzz_seed-5,
__consumer_offsets-39, fuzz_seed-44, __consumer_offsets-12, fuzz_seed-28,
fuzz_seed-24, __consumer_offsets-45, __consumer_offsets-1,
__consumer_offsets-5, __consumer_offsets-26, __consumer_offsets-29,
__consumer_offsets-34, __consumer_offsets-10, __consumer_offsets-32,
fuzz_seed-7, __consumer_offsets-40, fuzz_seed-15)
(kafka.server.ReplicaFetcherManager)
[2021-04-21 10:27:59,128] INFO [ReplicaAlterLogDirsManager on broker 0]
Removed fetcher for partitions HashSet(fuzz_seed-36, __consumer_offsets-35,
__consumer_offsets-37, fuzz_seed-42, fuzz_seed-34, fuzz_seed-1,
__consumer_offsets-22, fuzz_seed-17, __consumer_offsets-30, fuzz_seed-12,
fuzz_seed-47, fuzz_seed-27, __consumer_offsets-8, __consumer_offsets-21,
__consumer_offsets-4, __consumer_offsets-27, __consumer_offsets-7,
__consumer_offsets-9, __consumer_offsets-46, fuzz_seed-33, fuzz_seed-2,
fuzz_seed-18, fuzz_seed-0, fuzz_seed-43, __consumer_offsets-25,
__consumer_offsets-41, __consumer_offsets-23, __consumer_offsets-49,
fuzz_seed-26, fuzz_seed-19, __consumer_offsets-33, fuzz_seed-30,
fuzz_seed-39, fuzz_seed-14, __consumer_offsets-47, __consumer_offsets-16,
fuzz_seed-3, fuzz_seed-48, __consumer_offsets-28, fuzz_seed-13,
__consumer_offsets-31, __consumer_offsets-36, __consumer_offsets-42,
__consumer_offsets-3, __consumer_offsets-18, fuzz_seed-46, fuzz_seed-31,
fuzz_seed-9, fuzz_seed-35, __consumer_offsets-15, __consumer_offsets-24,
fuzz_seed-29, fuzz_seed-4, fuzz_seed-20, fuzz_seed-23,
__consumer_offsets-38, __consumer_offsets-48, fuzz_seed-45,
__consumer_offsets-17, fuzz_seed-49, fuzz_seed-40, __consumer_offsets-19,
__consumer_offsets-11, fuzz_seed-21, fuzz_seed-16, fuzz_seed-8,
__consumer_offsets-13, fuzz_seed-10, __consumer_offsets-2,
__consumer_offsets-43, __consumer_offsets-6, fuzz_seed-38,
__consumer_offsets-14, fuzz_seed-25, fuzz_seed-22, fuzz_seed-6,
fuzz_seed-41, __consumer_offsets-20, __consumer_offsets-0, fuzz_seed-37,
fuzz_seed-32, fuzz_seed-11, __consumer_offsets-44, fuzz_seed-5,
__consumer_offsets-39, fuzz_seed-44, __consumer_offsets-12, fuzz_seed-28,
fuzz_seed-24, __consumer_offsets-45, __consumer_offsets-1,
__consumer_offsets-5, __consumer_offsets-26, __consumer_offsets-29,
__consumer_offsets-34, __consumer_offsets-10, __consumer_offsets-32,
fuzz_seed-7, __consumer_offsets-40, fuzz_seed-15)
(kafka.server.ReplicaAlterLogDirsManager)
[2021-04-21 10:27:59,220] WARN [ReplicaManager broker=0] Broker 0 stopped
fetcher for partitions
fuzz_seed-36,__consumer_offsets-35,__consumer_offsets-37,fuzz_seed-42,fuzz_seed-34,fuzz_seed-1,__consumer_offsets-22,fuzz_seed-17,__consumer_offsets-30,fuzz_seed-12,fuzz_seed-47,fuzz_seed-27,__consumer_offsets-8,__consumer_offsets-21,__consumer_offsets-4,__consumer_offsets-27,__consumer_offsets-7,__consumer_offsets-9,__consumer_offsets-46,fuzz_seed-33,fuzz_seed-2,fuzz_seed-18,fuzz_seed-0,fuzz_seed-43,__consumer_offsets-25,__consumer_offsets-41,__consumer_offsets-23,__consumer_offsets-49,fuzz_seed-26,fuzz_seed-19,__consumer_offsets-33,fuzz_seed-30,fuzz_seed-39,fuzz_seed-14,__consumer_offsets-47,__consumer_offsets-16,fuzz_seed-3,fuzz_seed-48,__consumer_offsets-28,fuzz_seed-13,__consumer_offsets-31,__consumer_offsets-36,__consumer_offsets-42,__consumer_offsets-3,__consumer_offsets-18,fuzz_seed-46,fuzz_seed-31,fuzz_seed-9,fuzz_seed-35,__consumer_offsets-15,__consumer_offsets-24,fuzz_seed-29,fuzz_seed-4,fuzz_seed-20,fuzz_seed-23,__consumer_offsets-38,__consumer_offsets-48,fuzz_seed-45,__consumer_offsets-17,fuzz_seed-49,fuzz_seed-40,__consumer_offsets-19,__consumer_offsets-11,fuzz_seed-21,fuzz_seed-16,fuzz_seed-8,__consumer_offsets-13,fuzz_seed-10,__consumer_offsets-2,__consumer_offsets-43,__consumer_offsets-6,fuzz_seed-38,__consumer_offsets-14,fuzz_seed-25,fuzz_seed-22,fuzz_seed-6,fuzz_seed-41,__consumer_offsets-20,__consumer_offsets-0,fuzz_seed-37,fuzz_seed-32,fuzz_seed-11,__consumer_offsets-44,fuzz_seed-5,__consumer_offsets-39,fuzz_seed-44,__consumer_offsets-12,fuzz_seed-28,fuzz_seed-24,__consumer_offsets-45,__consumer_offsets-1,__consumer_offsets-5,__consumer_offsets-26,__consumer_offsets-29,__consumer_offsets-34,__consumer_offsets-10,__consumer_offsets-32,fuzz_seed-7,__consumer_offsets-40,fuzz_seed-15
and stopped moving logs for partitions because they are in the failed log
directory C:\Testing\.logskafka-logs. (kafka.server.ReplicaManager)
[2021-04-21 10:27:59,224] WARN Stopping serving logs in dir
C:\Testing\.logskafka-logs (kafka.log.LogManager)
[2021-04-21 10:27:59,232] ERROR Shutdown broker because all log dirs in
C:\Testing\.logskafka-logs have failed (kafka.log.LogManager)
Re: Kafkabroker log swap writing error
Posted by Shilin Wu <sh...@confluent.io.INVALID>.
Is the kafka for testing purposes?
Although it is cross platform, the best is still under linux so that os
buffer & cache can be efficient.
If you can, you could try to run under WSL2. See a tech blog here:
https://www.confluent.io/blog/set-up-and-run-kafka-on-windows-linux-wsl-2/
Lastly, if you have to run on windows as it is, you may try to find out
which process is holding the file handle.
https://serverfault.com/questions/1966/how-do-you-find-what-process-is-holding-a-file-open-in-windows
There might be another process holding it open so it can't be renamed.
(Unlike linux, windows doesn't allow that).
Thanks
Best Regards
Shilin
[image: Confluent] <https://www.confluent.io>
Wu Shilin
Solution Architect
+6581007012
Follow us: [image: Blog]
<https://www.confluent.io/blog?utm_source=footer&utm_medium=email&utm_campaign=ch.email-signature_type.community_content.blog>[image:
Twitter] <https://twitter.com/ConfluentInc>[image: LinkedIn]
<https://www.linkedin.com/company/confluent/>[image: Slack]
<https://slackpass.io/confluentcommunity>[image: YouTube]
<https://youtube.com/confluent>
[image: Kafka Summit] <https://www.kafka-summit.org/>
On Tue, Jun 15, 2021 at 6:08 AM Peter Lee <le...@gmail.com> wrote:
> ---------- Forwarded message ---------
> 보낸사람: hello <le...@gmail.com>
> Date: 2021년 6월 9일 (수) 오후 2:50
> Subject: Kafkabroker log swap writing error
> To: users-sc.1623217635.lobeokoidbfekkpndnbk-leepeter2019=
> gmail.com@kafka.apache.org
> <users-sc.1623217635.lobeokoidbfekkpndnbk-leepeter2019=
> gmail.com@kafka.apache.org>
>
>
> Whenever I run Kafka server in windows 10, it failed to rename
> controller.log to controller.log.time
>
>
>
> After several failed attempts, it stops running.
>
>
>
> *java.nio.file.FileSystemException:
>
> C:\Testing\.logskafka-logs\__consumer_offsets-31\00000000000000000000.index.cleaned
> ->
>
> C:\Testing\.logskafka-logs\__consumer_offsets-31\00000000000000000000.index.swap:*
>
> *“The process cannot access, because the other process is in use”*
>
>
>
> *[2021-04-21 10:27:59,232] ERROR Shutdown broker because all log dirs in
> C:\Testing\.logskafka-logs have failed (kafka.log.LogManager*
>
>
>
> Do you have any idea how to fix this?
>
>
>
>
>
> [2021-04-20 15:47:53,282] INFO [GroupCoordinator 0]: Stabilized group
> monitoragent generation 9 (__consumer_offsets-31)
> (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-20 15:47:59,474] INFO [GroupCoordinator 0]: Preparing to rebalance
> group monitoragent in state PreparingRebalance with old generation 9
> (__consumer_offsets-31) (reason: Adding new member
> node1-agent8-16cc2029-7ee0-4560-9890-028fea05b8b2 with group instance id
> None) (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-20 15:52:53,286] INFO [GroupCoordinator 0]: Member
> node1-agent8-4e7ee271-581a-40b2-9b1f-e5fb4f364704 in group monitoragent has
> failed, removing it from the group
> (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-20 15:52:53,287] INFO [GroupCoordinator 0]: Stabilized group
> monitoragent generation 10 (__consumer_offsets-31)
> (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-20 15:52:53,295] INFO [GroupCoordinator 0]: Assignment received
> from leader for group monitoragent for generation 10
> (kafka.coordinator.group.GroupCoordinator)
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-15].
>
> [2021-04-20 16:06:59,437] INFO [GroupCoordinator 0]: Member
> node1-agent8-16cc2029-7ee0-4560-9890-028fea05b8b2 in group monitoragent has
> failed, removing it from the group
> (kafka.coordinator.group.GroupCoordinator)
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/server.log] to
> [C:\kafka_2.13-2.7.0/logs/server.log.2021-04-20-15].
>
> [2021-04-20 16:06:59,438] INFO [GroupCoordinator 0]: Preparing to rebalance
> group monitoragent in state PreparingRebalance with old generation 10
> (__consumer_offsets-31) (reason: removing member
> node1-agent8-16cc2029-7ee0-4560-9890-028fea05b8b2 on heartbeat expiration)
> (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-20 16:06:59,439] INFO [GroupCoordinator 0]: Group monitoragent
> with generation 11 is now empty (__consumer_offsets-31)
> (kafka.coordinator.group.GroupCoordinator)
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-16].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-17].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-18].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-19].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-20].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-21].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-22].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-23].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-00].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-01].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-02].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-03].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-04].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-05].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-06].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-07].
>
> [2021-04-21 08:11:56,092] INFO [GroupCoordinator 0]: Preparing to rebalance
> group monitoragent in state PreparingRebalance with old generation 11
> (__consumer_offsets-31) (reason: Adding new member
> node1-agent8-bd039ca4-dca7-43a9-bd31-48b6a9d86031 with group instance id
> None) (kafka.coordinator.group.GroupCoordinator)
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/server.log] to
> [C:\kafka_2.13-2.7.0/logs/server.log.2021-04-20-16].
>
> [2021-04-21 08:11:56,095] INFO [GroupCoordinator 0]: Stabilized group
> monitoragent generation 12 (__consumer_offsets-31)
> (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-21 08:11:56,100] INFO [GroupCoordinator 0]: Assignment received
> from leader for group monitoragent for generation 12
> (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-21 08:16:46,832] INFO [GroupCoordinator 0]: Preparing to rebalance
> group monitoragent in state PreparingRebalance with old generation 12
> (__consumer_offsets-31) (reason: Adding new member
> node1-agent8-4a3f8f6f-1091-4954-99c1-d1ff2c8ae12e with group instance id
> None) (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-21 08:20:22,323] INFO [GroupCoordinator 0]: Member
> node1-agent8-bd039ca4-dca7-43a9-bd31-48b6a9d86031 in group monitoragent has
> failed, removing it from the group
> (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-21 08:20:22,326] INFO [GroupCoordinator 0]: Stabilized group
> monitoragent generation 13 (__consumer_offsets-31)
> (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-21 08:20:22,383] INFO [GroupCoordinator 0]: Assignment received
> from leader for group monitoragent for generation 13
> (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-21 08:23:38,927] INFO [GroupCoordinator 0]: Preparing to rebalance
> group monitoragent in state PreparingRebalance with old generation 13
> (__consumer_offsets-31) (reason: Adding new member
> node2-agent9-2210cbbe-9324-4d53-9b0e-a37997275127 with group instance id
> None) (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-21 08:26:23,712] INFO [GroupCoordinator 0]: Stabilized group
> monitoragent generation 14 (__consumer_offsets-31)
> (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-21 08:26:23,722] INFO [GroupCoordinator 0]: Assignment received
> from leader for group monitoragent for generation 14
> (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-21 09:01:02,878] INFO [GroupCoordinator 0]: Member[
> group.instance.id None, member.id
> node1-agent8-4a3f8f6f-1091-4954-99c1-d1ff2c8ae12e]
> in group monitoragent has left, removing it from the group
> (kafka.coordinator.group.GroupCoordinator)
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/server.log] to
> [C:\kafka_2.13-2.7.0/logs/server.log.2021-04-21-08].
>
> [2021-04-21 09:01:02,880] INFO [GroupCoordinator 0]: Preparing to rebalance
> group monitoragent in state PreparingRebalance with old generation 14
> (__consumer_offsets-31) (reason: removing member
> node1-agent8-4a3f8f6f-1091-4954-99c1-d1ff2c8ae12e on LeaveGroup)
> (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-21 09:02:27,729] INFO [GroupCoordinator 0]: Stabilized group
> monitoragent generation 15 (__consumer_offsets-31)
> (kafka.coordinator.group.GroupCoordinator)
>
> [2021-04-21 09:02:27,786] INFO [GroupCoordinator 0]: Assignment received
> from leader for group monitoragent for generation 15
> (kafka.coordinator.group.GroupCoordinator)
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-08].
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to
> [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-09].
>
> [2021-04-21 10:27:53,405] INFO [ProducerStateManager
> partition=__consumer_offsets-31] Writing producer snapshot at offset 70374
> (kafka.log.ProducerStateManager)
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/server.log] to
> [C:\kafka_2.13-2.7.0/logs/server.log.2021-04-21-09].
>
> [2021-04-21 10:27:53,412] INFO [Log partition=__consumer_offsets-31,
> dir=C:\Testing\.logskafka-logs] Rolled new log segment at offset 70374 in
> 11 ms. (kafka.log.Log)
>
> log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/log-cleaner.log] to
> [C:\kafka_2.13-2.7.0/logs/log-cleaner.log.2021-04-20-15].
>
> [2021-04-21 10:27:59,102] ERROR Failed to clean up log for
> __consumer_offsets-31 in dir C:\Testing\.logskafka-logs due to IOException
> (kafka.server.LogDirFailureChannel)
>
> java.nio.file.FileSystemException:
>
> C:\Testing\.logskafka-logs\__consumer_offsets-31\00000000000000000000.index.cleaned
> ->
>
> C:\Testing\.logskafka-logs\__consumer_offsets-31\00000000000000000000.index.swap:
> “The process cannot access, because the other process is in use”
>
>
>
> at sun.nio.fs.WindowsException.translateToIOException(Unknown
> Source)
>
> at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
>
> at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
>
> at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source)
>
> at java.nio.file.Files.move(Unknown Source)
>
> at
> org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:917)
>
> at kafka.log.AbstractIndex.renameTo(AbstractIndex.scala:211)
>
> at kafka.log.LazyIndex$IndexValue.renameTo(LazyIndex.scala:155)
>
> at kafka.log.LazyIndex.$anonfun$renameTo$1(LazyIndex.scala:79)
>
> at kafka.log.LazyIndex.renameTo(LazyIndex.scala:79)
>
> at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:496)
>
> at kafka.log.Log.$anonfun$replaceSegments$4(Log.scala:2338)
>
> at kafka.log.Log.$anonfun$replaceSegments$4$adapted(Log.scala:2338)
>
> at scala.collection.immutable.List.foreach(List.scala:333)
>
> at kafka.log.Log.replaceSegments(Log.scala:2338)
>
> at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:614)
>
> at kafka.log.Cleaner.$anonfun$doClean$6(LogCleaner.scala:539)
>
> at kafka.log.Cleaner.doClean(LogCleaner.scala:538)
>
> nsumer_offsets-31\00000000000000000000.index.swap: “The process cannot
> access, because the other process is in use”
>
>
>
> at
> sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
>
> at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown
> Source)
>
> at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
>
> at sun.nio.fs.WindowsFileSystemProvider.move(Unknown
> Source)
>
> at java.nio.file.Files.move(Unknown Source)
>
> at
> org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:914)
>
> ... 18 more
>
> [2021-04-21 10:27:59,120] WARN [ReplicaManager broker=0] Stopping serving
> replicas in dir C:\Testing\.logskafka-logs (kafka.server.ReplicaManager)
>
> [2021-04-21 10:27:59,124] INFO [ReplicaFetcherManager on broker 0] Removed
> fetcher for partitions HashSet(fuzz_seed-36, __consumer_offsets-35,
> __consumer_offsets-37, fuzz_seed-42, fuzz_seed-34, fuzz_seed-1,
> __consumer_offsets-22, fuzz_seed-17, __consumer_offsets-30, fuzz_seed-12,
> fuzz_seed-47, fuzz_seed-27, __consumer_offsets-8, __consumer_offsets-21,
> __consumer_offsets-4, __consumer_offsets-27, __consumer_offsets-7,
> __consumer_offsets-9, __consumer_offsets-46, fuzz_seed-33, fuzz_seed-2,
> fuzz_seed-18, fuzz_seed-0, fuzz_seed-43, __consumer_offsets-25,
> __consumer_offsets-41, __consumer_offsets-23, __consumer_offsets-49,
> fuzz_seed-26, fuzz_seed-19, __consumer_offsets-33, fuzz_seed-30,
> fuzz_seed-39, fuzz_seed-14, __consumer_offsets-47, __consumer_offsets-16,
> fuzz_seed-3, fuzz_seed-48, __consumer_offsets-28, fuzz_seed-13,
> __consumer_offsets-31, __consumer_offsets-36, __consumer_offsets-42,
> __consumer_offsets-3, __consumer_offsets-18, fuzz_seed-46, fuzz_seed-31,
> fuzz_seed-9, fuzz_seed-35, __consumer_offsets-15, __consumer_offsets-24,
> fuzz_seed-29, fuzz_seed-4, fuzz_seed-20, fuzz_seed-23,
> __consumer_offsets-38, __consumer_offsets-48, fuzz_seed-45,
> __consumer_offsets-17, fuzz_seed-49, fuzz_seed-40, __consumer_offsets-19,
> __consumer_offsets-11, fuzz_seed-21, fuzz_seed-16, fuzz_seed-8,
> __consumer_offsets-13, fuzz_seed-10, __consumer_offsets-2,
> __consumer_offsets-43, __consumer_offsets-6, fuzz_seed-38,
> __consumer_offsets-14, fuzz_seed-25, fuzz_seed-22, fuzz_seed-6,
> fuzz_seed-41, __consumer_offsets-20, __consumer_offsets-0, fuzz_seed-37,
> fuzz_seed-32, fuzz_seed-11, __consumer_offsets-44, fuzz_seed-5,
> __consumer_offsets-39, fuzz_seed-44, __consumer_offsets-12, fuzz_seed-28,
> fuzz_seed-24, __consumer_offsets-45, __consumer_offsets-1,
> __consumer_offsets-5, __consumer_offsets-26, __consumer_offsets-29,
> __consumer_offsets-34, __consumer_offsets-10, __consumer_offsets-32,
> fuzz_seed-7, __consumer_offsets-40, fuzz_seed-15)
> (kafka.server.ReplicaFetcherManager)
>
> [2021-04-21 10:27:59,128] INFO [ReplicaAlterLogDirsManager on broker 0]
> Removed fetcher for partitions HashSet(fuzz_seed-36, __consumer_offsets-35,
> __consumer_offsets-37, fuzz_seed-42, fuzz_seed-34, fuzz_seed-1,
> __consumer_offsets-22, fuzz_seed-17, __consumer_offsets-30, fuzz_seed-12,
> fuzz_seed-47, fuzz_seed-27, __consumer_offsets-8, __consumer_offsets-21,
> __consumer_offsets-4, __consumer_offsets-27, __consumer_offsets-7,
> __consumer_offsets-9, __consumer_offsets-46, fuzz_seed-33, fuzz_seed-2,
> fuzz_seed-18, fuzz_seed-0, fuzz_seed-43, __consumer_offsets-25,
> __consumer_offsets-41, __consumer_offsets-23, __consumer_offsets-49,
> fuzz_seed-26, fuzz_seed-19, __consumer_offsets-33, fuzz_seed-30,
> fuzz_seed-39, fuzz_seed-14, __consumer_offsets-47, __consumer_offsets-16,
> fuzz_seed-3, fuzz_seed-48, __consumer_offsets-28, fuzz_seed-13,
> __consumer_offsets-31, __consumer_offsets-36, __consumer_offsets-42,
> __consumer_offsets-3, __consumer_offsets-18, fuzz_seed-46, fuzz_seed-31,
> fuzz_seed-9, fuzz_seed-35, __consumer_offsets-15, __consumer_offsets-24,
> fuzz_seed-29, fuzz_seed-4, fuzz_seed-20, fuzz_seed-23,
> __consumer_offsets-38, __consumer_offsets-48, fuzz_seed-45,
> __consumer_offsets-17, fuzz_seed-49, fuzz_seed-40, __consumer_offsets-19,
> __consumer_offsets-11, fuzz_seed-21, fuzz_seed-16, fuzz_seed-8,
> __consumer_offsets-13, fuzz_seed-10, __consumer_offsets-2,
> __consumer_offsets-43, __consumer_offsets-6, fuzz_seed-38,
> __consumer_offsets-14, fuzz_seed-25, fuzz_seed-22, fuzz_seed-6,
> fuzz_seed-41, __consumer_offsets-20, __consumer_offsets-0, fuzz_seed-37,
> fuzz_seed-32, fuzz_seed-11, __consumer_offsets-44, fuzz_seed-5,
> __consumer_offsets-39, fuzz_seed-44, __consumer_offsets-12, fuzz_seed-28,
> fuzz_seed-24, __consumer_offsets-45, __consumer_offsets-1,
> __consumer_offsets-5, __consumer_offsets-26, __consumer_offsets-29,
> __consumer_offsets-34, __consumer_offsets-10, __consumer_offsets-32,
> fuzz_seed-7, __consumer_offsets-40, fuzz_seed-15)
> (kafka.server.ReplicaAlterLogDirsManager)
>
> [2021-04-21 10:27:59,220] WARN [ReplicaManager broker=0] Broker 0 stopped
> fetcher for partitions
>
> fuzz_seed-36,__consumer_offsets-35,__consumer_offsets-37,fuzz_seed-42,fuzz_seed-34,fuzz_seed-1,__consumer_offsets-22,fuzz_seed-17,__consumer_offsets-30,fuzz_seed-12,fuzz_seed-47,fuzz_seed-27,__consumer_offsets-8,__consumer_offsets-21,__consumer_offsets-4,__consumer_offsets-27,__consumer_offsets-7,__consumer_offsets-9,__consumer_offsets-46,fuzz_seed-33,fuzz_seed-2,fuzz_seed-18,fuzz_seed-0,fuzz_seed-43,__consumer_offsets-25,__consumer_offsets-41,__consumer_offsets-23,__consumer_offsets-49,fuzz_seed-26,fuzz_seed-19,__consumer_offsets-33,fuzz_seed-30,fuzz_seed-39,fuzz_seed-14,__consumer_offsets-47,__consumer_offsets-16,fuzz_seed-3,fuzz_seed-48,__consumer_offsets-28,fuzz_seed-13,__consumer_offsets-31,__consumer_offsets-36,__consumer_offsets-42,__consumer_offsets-3,__consumer_offsets-18,fuzz_seed-46,fuzz_seed-31,fuzz_seed-9,fuzz_seed-35,__consumer_offsets-15,__consumer_offsets-24,fuzz_seed-29,fuzz_seed-4,fuzz_seed-20,fuzz_seed-23,__consumer_offsets-38,__consumer_offsets-48,fuzz_seed-45,__consumer_offsets-17,fuzz_seed-49,fuzz_seed-40,__consumer_offsets-19,__consumer_offsets-11,fuzz_seed-21,fuzz_seed-16,fuzz_seed-8,__consumer_offsets-13,fuzz_seed-10,__consumer_offsets-2,__consumer_offsets-43,__consumer_offsets-6,fuzz_seed-38,__consumer_offsets-14,fuzz_seed-25,fuzz_seed-22,fuzz_seed-6,fuzz_seed-41,__consumer_offsets-20,__consumer_offsets-0,fuzz_seed-37,fuzz_seed-32,fuzz_seed-11,__consumer_offsets-44,fuzz_seed-5,__consumer_offsets-39,fuzz_seed-44,__consumer_offsets-12,fuzz_seed-28,fuzz_seed-24,__consumer_offsets-45,__consumer_offsets-1,__consumer_offsets-5,__consumer_offsets-26,__consumer_offsets-29,__consumer_offsets-34,__consumer_offsets-10,__consumer_offsets-32,fuzz_seed-7,__consumer_offsets-40,fuzz_seed-15
> and stopped moving logs for partitions because they are in the failed log
> directory C:\Testing\.logskafka-logs. (kafka.server.ReplicaManager)
>
> [2021-04-21 10:27:59,224] WARN Stopping serving logs in dir
> C:\Testing\.logskafka-logs (kafka.log.LogManager)
>
> [2021-04-21 10:27:59,232] ERROR Shutdown broker because all log dirs in
> C:\Testing\.logskafka-logs have failed (kafka.log.LogManager)
>