You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2020/09/11 15:09:33 UTC
[GitHub] [incubator-iotdb] yuqi1129 opened a new pull request #1727: [IOTDB-888] Refactor `SingleFileLogReader` iterator logic
yuqi1129 opened a new pull request #1727:
URL: https://github.com/apache/incubator-iotdb/pull/1727
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] yuqi1129 commented on a change in pull request #1727: [IOTDB-888] Refactor `SingleFileLogReader` iterator logic
Posted by GitBox <gi...@apache.org>.
yuqi1129 commented on a change in pull request #1727:
URL: https://github.com/apache/incubator-iotdb/pull/1727#discussion_r488599312
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
Review comment:
Yes, if `hasNext()` should be called, there is not need to make any changes here
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
+ hasNext = batchLogReader.hasNext();
- int readLen = logStream.read(buffer, 0, logSize);
- if (readLen < logSize) {
- throw new IOException("Reach eof");
+ if (hasNext) {
+ return hasNext;
}
+ }
- final long checkSum = logStream.readLong();
- checkSummer.reset();
- checkSummer.update(buffer, 0, logSize);
- if (checkSummer.getValue() != checkSum) {
- throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
- + "file: "
- + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ try {
+ while (logStream.available() > LEAST_LOG_SIZE) {
Review comment:
if `batchLogReader ` is null, then we should find a position in logStream where we create BatchLogReader and make `hasNext()` is true repeatedly, it's in fact a loop
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] yuqi1129 commented on a change in pull request #1727: [IOTDB-888] Refactor `SingleFileLogReader` iterator logic
Posted by GitBox <gi...@apache.org>.
yuqi1129 commented on a change in pull request #1727:
URL: https://github.com/apache/incubator-iotdb/pull/1727#discussion_r488594195
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
+ hasNext = batchLogReader.hasNext();
- int readLen = logStream.read(buffer, 0, logSize);
- if (readLen < logSize) {
- throw new IOException("Reach eof");
+ if (hasNext) {
+ return hasNext;
}
+ }
- final long checkSum = logStream.readLong();
- checkSummer.reset();
- checkSummer.update(buffer, 0, logSize);
- if (checkSummer.getValue() != checkSum) {
- throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
- + "file: "
- + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ try {
+ while (logStream.available() > LEAST_LOG_SIZE) {
+ int logSize = logStream.readInt();
+ if (logSize <= 0) {
+ hasNext = false;
+ return hasNext;
+ }
+
+ byte[] buffer = new byte[logSize];
+ int readLen = logStream.read(buffer, 0, logSize);
+ if (readLen < logSize) {
+ throw new IOException("Reach eof");
+ }
+
+ final long checkSum = logStream.readLong();
+ checkSummer.reset();
+ checkSummer.update(buffer, 0, logSize);
+ if (checkSummer.getValue() != checkSum) {
+ throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
+ + "file: "
+ + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ }
+
+ batchLogReader = new BatchLogReader(ByteBuffer.wrap(buffer));
+ if (batchLogReader.isFileCorrupted()) {
+ fileCorrupted = true;
+ hasNext = false;
+ return hasNext;
+ }
+
+ if (batchLogReader.hasNext()) {
+ hasNext = true;
+ return hasNext;
+ }
}
-
- batchLogReader = new BatchLogReader(ByteBuffer.wrap(buffer));
- fileCorrupted = fileCorrupted || batchLogReader.isFileCorrupted();
} catch (Exception e) {
logger.error("Cannot read more PhysicalPlans from {} because", filepath, e);
- fileCorrupted = true;
- return false;
}
- return true;
+
+ hasNext = false;
+ return hasNext;
}
@Override
public PhysicalPlan next() {
- if (!hasNext()){
+ if (!hasNext){
Review comment:
You are right, we should invoke `hasNext()` when we call next(), i change the logic here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] jixuan1989 commented on a change in pull request #1727: [IOTDB-888] Refactor `SingleFileLogReader` iterator logic
Posted by GitBox <gi...@apache.org>.
jixuan1989 commented on a change in pull request #1727:
URL: https://github.com/apache/incubator-iotdb/pull/1727#discussion_r487414891
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
Review comment:
IMO, keep `if (batchLogReader != null && batchLogReader.hasNext())` is better...
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
+ hasNext = batchLogReader.hasNext();
- int readLen = logStream.read(buffer, 0, logSize);
- if (readLen < logSize) {
- throw new IOException("Reach eof");
+ if (hasNext) {
+ return hasNext;
}
+ }
- final long checkSum = logStream.readLong();
- checkSummer.reset();
- checkSummer.update(buffer, 0, logSize);
- if (checkSummer.getValue() != checkSum) {
- throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
- + "file: "
- + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ try {
+ while (logStream.available() > LEAST_LOG_SIZE) {
Review comment:
why add `while` here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] jixuan1989 commented on pull request #1727: [IOTDB-888] Refactor `SingleFileLogReader` iterator logic
Posted by GitBox <gi...@apache.org>.
jixuan1989 commented on pull request #1727:
URL: https://github.com/apache/incubator-iotdb/pull/1727#issuecomment-691387514
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] jixuan1989 commented on a change in pull request #1727: [IOTDB-888] Refactor `SingleFileLogReader` iterator logic
Posted by GitBox <gi...@apache.org>.
jixuan1989 commented on a change in pull request #1727:
URL: https://github.com/apache/incubator-iotdb/pull/1727#discussion_r487414891
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
Review comment:
IMO, keep `if (batchLogReader != null && batchLogReader.hasNext())` is better...
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
+ hasNext = batchLogReader.hasNext();
- int readLen = logStream.read(buffer, 0, logSize);
- if (readLen < logSize) {
- throw new IOException("Reach eof");
+ if (hasNext) {
+ return hasNext;
}
+ }
- final long checkSum = logStream.readLong();
- checkSummer.reset();
- checkSummer.update(buffer, 0, logSize);
- if (checkSummer.getValue() != checkSum) {
- throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
- + "file: "
- + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ try {
+ while (logStream.available() > LEAST_LOG_SIZE) {
Review comment:
why add `while` here?
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
Review comment:
IMO, keep `if (batchLogReader != null && batchLogReader.hasNext())` is better...
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
+ hasNext = batchLogReader.hasNext();
- int readLen = logStream.read(buffer, 0, logSize);
- if (readLen < logSize) {
- throw new IOException("Reach eof");
+ if (hasNext) {
+ return hasNext;
}
+ }
- final long checkSum = logStream.readLong();
- checkSummer.reset();
- checkSummer.update(buffer, 0, logSize);
- if (checkSummer.getValue() != checkSum) {
- throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
- + "file: "
- + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ try {
+ while (logStream.available() > LEAST_LOG_SIZE) {
Review comment:
why add `while` here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] jixuan1989 commented on a change in pull request #1727: [IOTDB-888] Refactor `SingleFileLogReader` iterator logic
Posted by GitBox <gi...@apache.org>.
jixuan1989 commented on a change in pull request #1727:
URL: https://github.com/apache/incubator-iotdb/pull/1727#discussion_r487414891
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
Review comment:
IMO, keep `if (batchLogReader != null && batchLogReader.hasNext())` is better...
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
+ hasNext = batchLogReader.hasNext();
- int readLen = logStream.read(buffer, 0, logSize);
- if (readLen < logSize) {
- throw new IOException("Reach eof");
+ if (hasNext) {
+ return hasNext;
}
+ }
- final long checkSum = logStream.readLong();
- checkSummer.reset();
- checkSummer.update(buffer, 0, logSize);
- if (checkSummer.getValue() != checkSum) {
- throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
- + "file: "
- + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ try {
+ while (logStream.available() > LEAST_LOG_SIZE) {
Review comment:
why add `while` here?
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
Review comment:
IMO, keep `if (batchLogReader != null && batchLogReader.hasNext())` is better...
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
+ hasNext = batchLogReader.hasNext();
- int readLen = logStream.read(buffer, 0, logSize);
- if (readLen < logSize) {
- throw new IOException("Reach eof");
+ if (hasNext) {
+ return hasNext;
}
+ }
- final long checkSum = logStream.readLong();
- checkSummer.reset();
- checkSummer.update(buffer, 0, logSize);
- if (checkSummer.getValue() != checkSum) {
- throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
- + "file: "
- + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ try {
+ while (logStream.available() > LEAST_LOG_SIZE) {
Review comment:
why add `while` here?
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
Review comment:
IMO, keep `if (batchLogReader != null && batchLogReader.hasNext())` is better...
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
+ hasNext = batchLogReader.hasNext();
- int readLen = logStream.read(buffer, 0, logSize);
- if (readLen < logSize) {
- throw new IOException("Reach eof");
+ if (hasNext) {
+ return hasNext;
}
+ }
- final long checkSum = logStream.readLong();
- checkSummer.reset();
- checkSummer.update(buffer, 0, logSize);
- if (checkSummer.getValue() != checkSum) {
- throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
- + "file: "
- + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ try {
+ while (logStream.available() > LEAST_LOG_SIZE) {
Review comment:
why add `while` here?
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
Review comment:
IMO, keep `if (batchLogReader != null && batchLogReader.hasNext())` is better...
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
+ hasNext = batchLogReader.hasNext();
- int readLen = logStream.read(buffer, 0, logSize);
- if (readLen < logSize) {
- throw new IOException("Reach eof");
+ if (hasNext) {
+ return hasNext;
}
+ }
- final long checkSum = logStream.readLong();
- checkSummer.reset();
- checkSummer.update(buffer, 0, logSize);
- if (checkSummer.getValue() != checkSum) {
- throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
- + "file: "
- + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ try {
+ while (logStream.available() > LEAST_LOG_SIZE) {
Review comment:
why add `while` here?
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
Review comment:
IMO, keep `if (batchLogReader != null && batchLogReader.hasNext())` is better...
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
+ hasNext = batchLogReader.hasNext();
- int readLen = logStream.read(buffer, 0, logSize);
- if (readLen < logSize) {
- throw new IOException("Reach eof");
+ if (hasNext) {
+ return hasNext;
}
+ }
- final long checkSum = logStream.readLong();
- checkSummer.reset();
- checkSummer.update(buffer, 0, logSize);
- if (checkSummer.getValue() != checkSum) {
- throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
- + "file: "
- + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ try {
+ while (logStream.available() > LEAST_LOG_SIZE) {
Review comment:
why add `while` here?
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
Review comment:
IMO, keep `if (batchLogReader != null && batchLogReader.hasNext())` is better...
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
+ hasNext = batchLogReader.hasNext();
- int readLen = logStream.read(buffer, 0, logSize);
- if (readLen < logSize) {
- throw new IOException("Reach eof");
+ if (hasNext) {
+ return hasNext;
}
+ }
- final long checkSum = logStream.readLong();
- checkSummer.reset();
- checkSummer.update(buffer, 0, logSize);
- if (checkSummer.getValue() != checkSum) {
- throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
- + "file: "
- + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ try {
+ while (logStream.available() > LEAST_LOG_SIZE) {
Review comment:
why add `while` here?
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
Review comment:
IMO, keep `if (batchLogReader != null && batchLogReader.hasNext())` is better...
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
+ hasNext = batchLogReader.hasNext();
- int readLen = logStream.read(buffer, 0, logSize);
- if (readLen < logSize) {
- throw new IOException("Reach eof");
+ if (hasNext) {
+ return hasNext;
}
+ }
- final long checkSum = logStream.readLong();
- checkSummer.reset();
- checkSummer.update(buffer, 0, logSize);
- if (checkSummer.getValue() != checkSum) {
- throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
- + "file: "
- + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ try {
+ while (logStream.available() > LEAST_LOG_SIZE) {
Review comment:
why add `while` here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] jixuan1989 commented on pull request #1727: [IOTDB-888] Refactor `SingleFileLogReader` iterator logic
Posted by GitBox <gi...@apache.org>.
jixuan1989 commented on pull request #1727:
URL: https://github.com/apache/incubator-iotdb/pull/1727#issuecomment-691387514
thanks @yuqi1129 .
issue #1726 @jt2594838 , can you have a review?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #1727: [IOTDB-888] Refactor `SingleFileLogReader` iterator logic
Posted by GitBox <gi...@apache.org>.
jt2594838 commented on a change in pull request #1727:
URL: https://github.com/apache/incubator-iotdb/pull/1727#discussion_r487717421
##########
File path: server/src/main/java/org/apache/iotdb/db/writelog/io/SingleFileLogReader.java
##########
@@ -53,54 +54,69 @@
private boolean fileCorrupted = false;
+ private boolean hasNext = false;
+
public SingleFileLogReader(File logFile) throws FileNotFoundException {
open(logFile);
}
@Override
public boolean hasNext() {
- try {
- if (batchLogReader != null && batchLogReader.hasNext()) {
- return true;
- }
-
- if (logStream.available() < LEAST_LOG_SIZE) {
- return false;
- }
- int logSize = logStream.readInt();
- if (logSize <= 0) {
- return false;
- }
- buffer = new byte[logSize];
+ if (Objects.nonNull(batchLogReader)) {
+ hasNext = batchLogReader.hasNext();
- int readLen = logStream.read(buffer, 0, logSize);
- if (readLen < logSize) {
- throw new IOException("Reach eof");
+ if (hasNext) {
+ return hasNext;
}
+ }
- final long checkSum = logStream.readLong();
- checkSummer.reset();
- checkSummer.update(buffer, 0, logSize);
- if (checkSummer.getValue() != checkSum) {
- throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
- + "file: "
- + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ try {
+ while (logStream.available() > LEAST_LOG_SIZE) {
+ int logSize = logStream.readInt();
+ if (logSize <= 0) {
+ hasNext = false;
+ return hasNext;
+ }
+
+ byte[] buffer = new byte[logSize];
+ int readLen = logStream.read(buffer, 0, logSize);
+ if (readLen < logSize) {
+ throw new IOException("Reach eof");
+ }
+
+ final long checkSum = logStream.readLong();
+ checkSummer.reset();
+ checkSummer.update(buffer, 0, logSize);
+ if (checkSummer.getValue() != checkSum) {
+ throw new IOException(String.format("The check sum of the No.%d log batch is incorrect! In "
+ + "file: "
+ + "%d Calculated: %d.", idx, checkSum, checkSummer.getValue()));
+ }
+
+ batchLogReader = new BatchLogReader(ByteBuffer.wrap(buffer));
+ if (batchLogReader.isFileCorrupted()) {
+ fileCorrupted = true;
+ hasNext = false;
+ return hasNext;
+ }
+
+ if (batchLogReader.hasNext()) {
+ hasNext = true;
+ return hasNext;
+ }
}
-
- batchLogReader = new BatchLogReader(ByteBuffer.wrap(buffer));
- fileCorrupted = fileCorrupted || batchLogReader.isFileCorrupted();
} catch (Exception e) {
logger.error("Cannot read more PhysicalPlans from {} because", filepath, e);
- fileCorrupted = true;
- return false;
}
- return true;
+
+ hasNext = false;
+ return hasNext;
}
@Override
public PhysicalPlan next() {
- if (!hasNext()){
+ if (!hasNext){
Review comment:
`next()` in an iterator should not depend on `hasNext()`, that is, the caller should be able to iterate without calling `hasNext()`. While in your implementation, if the caller calls `next()` without calling `hasNext()`, it will get nothing at all.
And I do not understand other code changes in this PR, could you please explain their improvements to me?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] jixuan1989 commented on pull request #1727: [IOTDB-888] Refactor `SingleFileLogReader` iterator logic
Posted by GitBox <gi...@apache.org>.
jixuan1989 commented on pull request #1727:
URL: https://github.com/apache/incubator-iotdb/pull/1727#issuecomment-691387514
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org