You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2021/10/09 02:07:33 UTC
[GitHub] [rocketmq] charlieDJ opened a new issue #3401: delayOffset.json and consumerOffset were corrupted when windows system crashes unexpectedly
charlieDJ opened a new issue #3401:
URL: https://github.com/apache/rocketmq/issues/3401
OS: Windows Server 2008 R2
RocketMQ release version: 4.9.1
This happens from time to time. I restart my PC and I can't start the broker after windows system crashes unexpectedly. Then I checked the broker logs. It shows that delayOffset.json and consumerOffset.json were corrupted. Human cannot read the text. Then I delete all the characters and rewrite the json data. I restart the broker, it works.
But all the offsets were lost, the messages were recomsumed. I'm not sure whethere is it a bug. The problem haunts me for a long time. Can you provide a good reason to explain it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Jason918 commented on issue #3401: delayOffset.json and consumerOffset were corrupted when windows system crashes unexpectedly
Posted by GitBox <gi...@apache.org>.
Jason918 commented on issue #3401:
URL: https://github.com/apache/rocketmq/issues/3401#issuecomment-940592207
> It shows that delayOffset.json and consumerOffset.json were corrupted.
It's better that you can attach these corrupted files.
You track the implementation of persisting "consumerOffset.json" file in `org.apache.rocketmq.common.ConfigManager#persist`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] charlieDJ commented on issue #3401: delayOffset.json and consumerOffset were corrupted when windows system crashes unexpectedly
Posted by GitBox <gi...@apache.org>.
charlieDJ commented on issue #3401:
URL: https://github.com/apache/rocketmq/issues/3401#issuecomment-942841457
> > > The json data format is wrong, is there less closing brace?
> >
> >
> > ALL the data represented in other encoding. Characters look like XXXX. I can't find the corrupted file for the time being because I didn't make a backup.
>
> Is there topics.json.bak file in this directory?
It remained correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] charlieDJ commented on issue #3401: delayOffset.json and consumerOffset were corrupted when windows system crashes unexpectedly
Posted by GitBox <gi...@apache.org>.
charlieDJ commented on issue #3401:
URL: https://github.com/apache/rocketmq/issues/3401#issuecomment-942841897
> > It shows that delayOffset.json and consumerOffset.json were corrupted.
>
> It's better that you can attach these corrupted files.
>
> You track the implementation of persisting "consumerOffset.json" file in `org.apache.rocketmq.common.ConfigManager#persist`.
well, I'll keep oberserving.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] charlieDJ commented on issue #3401: delayOffset.json and consumerOffset were corrupted when windows system crashes unexpectedly
Posted by GitBox <gi...@apache.org>.
charlieDJ commented on issue #3401:
URL: https://github.com/apache/rocketmq/issues/3401#issuecomment-948204841
I can't find the file because I have changed my OS. Now it's stable. Everything works fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Jason918 commented on issue #3401: delayOffset.json and consumerOffset were corrupted when windows system crashes unexpectedly
Posted by GitBox <gi...@apache.org>.
Jason918 commented on issue #3401:
URL: https://github.com/apache/rocketmq/issues/3401#issuecomment-942881675
The persistence procedure of these meta data does not check checksum of the data.
I think we can optimize it by adding checksum of the data file, and revert to backup file if the checksum does not match.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] panzhi33 commented on issue #3401: delayOffset.json and consumerOffset were corrupted when windows system crashes unexpectedly
Posted by GitBox <gi...@apache.org>.
panzhi33 commented on issue #3401:
URL: https://github.com/apache/rocketmq/issues/3401#issuecomment-942890409
> delayOffset.json
you can post the damaged delayOffset.json to see
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] lizhimins commented on issue #3401: delayOffset.json and consumerOffset were corrupted when windows system crashes unexpectedly
Posted by GitBox <gi...@apache.org>.
lizhimins commented on issue #3401:
URL: https://github.com/apache/rocketmq/issues/3401#issuecomment-939420762
First of all, find the cause of the system crash
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] lizhiboo commented on issue #3401: delayOffset.json and consumerOffset were corrupted when windows system crashes unexpectedly
Posted by GitBox <gi...@apache.org>.
lizhiboo commented on issue #3401:
URL: https://github.com/apache/rocketmq/issues/3401#issuecomment-940569233
> > The json data format is wrong, is there less closing brace?
>
> ALL the data represented in other encoding. Characters look like XXXX. I can't find the corrupted file for the time being because I didn't make a backup.
Is there topics.json.bak file in this directory?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] charlieDJ commented on issue #3401: delayOffset.json and consumerOffset were corrupted when windows system crashes unexpectedly
Posted by GitBox <gi...@apache.org>.
charlieDJ commented on issue #3401:
URL: https://github.com/apache/rocketmq/issues/3401#issuecomment-939700623
> The json data format is wrong, is there less closing brace?
ALL the data represented in other encoding. Characters look like XXXX. I can't find the corrupted file for the time being because I didn't make a backup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] panzhi33 commented on issue #3401: delayOffset.json and consumerOffset were corrupted when windows system crashes unexpectedly
Posted by GitBox <gi...@apache.org>.
panzhi33 commented on issue #3401:
URL: https://github.com/apache/rocketmq/issues/3401#issuecomment-939211305
The json data format is wrong, is there less closing brace?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] charlieDJ commented on issue #3401: delayOffset.json and consumerOffset were corrupted when windows system crashes unexpectedly
Posted by GitBox <gi...@apache.org>.
charlieDJ commented on issue #3401:
URL: https://github.com/apache/rocketmq/issues/3401#issuecomment-942855338
> > It shows that delayOffset.json and consumerOffset.json were corrupted.
>
> It's better that you can attach these corrupted files.
>
> You track the implementation of persisting "consumerOffset.json" file in `org.apache.rocketmq.common.ConfigManager#persist`.
The power went out last night. I restart my VM now and find that the delayOffset.json file is corrupted. The offsetTable seems clearing out in consumerOffset.json file.
delayOffset.json file:
It looks like blank empty. It represents multiple NUL in Ansi encoding by default.
consumerOffset.json file:
{
"offsetTable":{}
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org