You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ratis.apache.org by GitBox <gi...@apache.org> on 2020/05/14 18:58:25 UTC
[GitHub] [incubator-ratis] hanishakoneru opened a new pull request #100: RATIS-937. Loading of open segments always throws corrupted segment warning
hanishakoneru opened a new pull request #100:
URL: https://github.com/apache/incubator-ratis/pull/100
When an open log segment is loaded, it always logs a warning that the segment is corrupted.
```
Segment file is corrupted: expected to have -43 entries but only 8 entries read successfully
```
LogSegment#loadSegment() has the following check for corruption:
```
final int expectedEntryCount = Math.toIntExact(end - start + 1);
final boolean corrupted = entryCount != expectedEntryCount;
if (corrupted) {
LOG.warn("Segment file is corrupted: expected to have {} entries but only {} entries read successfully",
expectedEntryCount, entryCount);
}
```
But the end is always INVALID_LOG_INDEX (-1) for an open segment.
Before this check, entries are appended to the segment. So the end variable should be updated with the correct endIndex of the segment before checking for corruption.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] runzhiwang commented on pull request #100: RATIS-937. Loading of open segments always throws corrupted segment warning
Posted by GitBox <gi...@apache.org>.
runzhiwang commented on pull request #100:
URL: https://github.com/apache/incubator-ratis/pull/100#issuecomment-629073695
+1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] mukul1987 commented on a change in pull request #100: RATIS-937. Loading of open segments always throws corrupted segment warning
Posted by GitBox <gi...@apache.org>.
mukul1987 commented on a change in pull request #100:
URL: https://github.com/apache/incubator-ratis/pull/100#discussion_r425827891
##########
File path: ratis-server/src/main/java/org/apache/ratis/server/raftlog/segmented/LogSegment.java
##########
@@ -145,6 +147,10 @@ static LogSegment loadSegment(RaftStorage storage, File file, long start, long e
});
LOG.info("Successfully read {} entries from segment file {}", entryCount, file);
+ if (isOpen && end == INVALID_LOG_INDEX) {
Review comment:
Also how is the segment's end index initialized after a node restart ?
##########
File path: ratis-server/src/main/java/org/apache/ratis/server/raftlog/segmented/LogSegment.java
##########
@@ -145,6 +147,10 @@ static LogSegment loadSegment(RaftStorage storage, File file, long start, long e
});
LOG.info("Successfully read {} entries from segment file {}", entryCount, file);
+ if (isOpen && end == INVALID_LOG_INDEX) {
Review comment:
If the segment is open, the end should always be INVALID_LOG_INDEX?
Are there cases where it is not INVALID_LOG_INDEX ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] hanishakoneru commented on a change in pull request #100: RATIS-937. Loading of open segments always throws corrupted segment warning
Posted by GitBox <gi...@apache.org>.
hanishakoneru commented on a change in pull request #100:
URL: https://github.com/apache/incubator-ratis/pull/100#discussion_r425924842
##########
File path: ratis-server/src/main/java/org/apache/ratis/server/raftlog/segmented/LogSegment.java
##########
@@ -145,6 +147,10 @@ static LogSegment loadSegment(RaftStorage storage, File file, long start, long e
});
LOG.info("Successfully read {} entries from segment file {}", entryCount, file);
+ if (isOpen && end == INVALID_LOG_INDEX) {
Review comment:
Yes it should always be INVALID_LOG_INDEX. I will remove that check.
Segment endIndex is updated during readSegmentFile call. It in turn calls segment.append for each entry and updates the endIndex.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] mukul1987 merged pull request #100: RATIS-937. Loading of open segments always throws corrupted segment warning
Posted by GitBox <gi...@apache.org>.
mukul1987 merged pull request #100:
URL: https://github.com/apache/incubator-ratis/pull/100
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org