Lots of warns about LogCleaningPaused during partition reassignment

[BGCH <ig...@hivehome.com>]

Hello!

We tried to migrate data from 0.10.2.1 cluster to 0.11.0.2. Firstly we 
spread topics to both clusters. There were lots of problems and restarts 
of some nodes of both clusters (we probably shouldn't do that). All this 
ended up with a state when we had lots of exceptions from 2 nodes of 
0.10 cluster:
/java.lang.IllegalStateException: Compaction for partition topic_name-7 
cannot be aborted and paused since it is in LogCleaningPaused state./

and the whole reassignment process stuck.

I looked through the source code of LogManager and found KAFKA-3123 
<https://issues.apache.org/jira/browse/KAFKA-3123> which may be the 
cause. I restarted that 2 nodes and reassignment proceeded, but now I have /
[ReplicaFetcherThread-0-1028], Error for partition [topic_name,33] to 
broker 
1028:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: 
This server does not host this topic-partition./

My guess is that Kafka could not roll out new segments of logs due to 
LogCleaningPaused exception and after restart some segments lost. Is 
this correct?

Is there any other possible cause of that /LogCleaningPaused /exception? 
And the main question is how to prevent all this?
//


-- 
hivehome.com <http://www.hivehome.com>



Hive | London | Cambridge | Houston | Toronto
The information contained in or attached to this email is confidential and 
intended only for the use of the individual(s) to which it is addressed. It 
may contain information which is confidential and/or covered by legal 
professional or other privilege. The views expressed in this email are not 
necessarily the views of Centrica plc, and the company, its directors, 
officers or employees make no representation or accept any liability for 
their accuracy or completeness unless expressly stated to the contrary. 
Centrica Connected Home Limited (company no: 5782908), registered in 
England and Wales with its registered office at Millstream, Maidenhead 
Road, Windsor, Berkshire SL4 5GD.