You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2020/10/23 07:35:42 UTC
[GitHub] [iotdb] wshao08 opened a new pull request #1851: [IOTDB-954] Fix auto-creating schema in parallel may write unrecognizable timeseries in mlog.txt
wshao08 opened a new pull request #1851:
URL: https://github.com/apache/iotdb/pull/1851
Jira Issue: https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-954?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] samperson1997 commented on a change in pull request #1851: [IOTDB-954] Fix auto-creating schema in parallel may write unrecognizable timeseries in mlog.txt
Posted by GitBox <gi...@apache.org>.
samperson1997 commented on a change in pull request #1851:
URL: https://github.com/apache/iotdb/pull/1851#discussion_r511570730
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/MLogWriter.java
##########
@@ -53,40 +58,50 @@ public MLogWriter(String schemaDir, String logFileName) throws IOException {
logFile = SystemFileFactory.INSTANCE.getFile(schemaDir + File.separator + logFileName);
FileWriter fileWriter = new FileWriter(logFile, true);
writer = new BufferedWriter(fileWriter);
+ fileOutputStream = new FileOutputStream(logFile, true);
+ channel = fileOutputStream.getChannel();
}
public void close() throws IOException {
writer.close();
+ channel.close();
+ fileOutputStream.close();
}
public void createTimeseries(CreateTimeSeriesPlan plan, long offset) throws IOException {
- writer.write(String.format("%s,%s,%s,%s,%s", MetadataOperationType.CREATE_TIMESERIES,
- plan.getPath().getFullPath(), plan.getDataType().serialize(),
- plan.getEncoding().serialize(), plan.getCompressor().serialize()));
+ StringBuilder buf = new StringBuilder();
+ buf.append(String.format("%s,%s,%s,%s,%s", MetadataOperationType.CREATE_TIMESERIES,
+ plan.getPath().getFullPath(), plan.getDataType().serialize(),
+ plan.getEncoding().serialize(), plan.getCompressor().serialize()));
- writer.write(",");
+ buf.append(",");
if (plan.getProps() != null) {
boolean first = true;
for (Map.Entry<String, String> entry : plan.getProps().entrySet()) {
if (first) {
- writer.write(String.format("%s=%s", entry.getKey(), entry.getValue()));
+ buf.append(String.format("%s=%s", entry.getKey(), entry.getValue()));
first = false;
} else {
- writer.write(String.format("&%s=%s", entry.getKey(), entry.getValue()));
+ buf.append(String.format("&%s=%s", entry.getKey(), entry.getValue()));
}
}
}
- writer.write(",");
+ buf.append(",");
if (plan.getAlias() != null) {
- writer.write(plan.getAlias());
+ buf.append(plan.getAlias());
}
- writer.write(",");
+ buf.append(",");
if (offset >= 0) {
- writer.write(String.valueOf(offset));
+ buf.append(offset);
}
- newLine();
+ final String newLine = System.getProperty("line.separator");
+ buf.append(newLine);
+
+ channel.write(ByteBuffer.wrap(buf.toString().getBytes()));
+ channel.force(true);
+ ++lineNumber;
}
public void deleteTimeseries(String path) throws IOException {
Review comment:
I think you could use `channel.write()` to implement all the functions in this class, including `deleteTimeseries`, `setStorageGroup`, `deleteStorageGroup`, `setTTL`, `changeOffset` and so on
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] SilverNarcissus commented on a change in pull request #1851: [IOTDB-951] Fix auto-creating schema in parallel may write unrecognizable timeseries in mlog.txt
Posted by GitBox <gi...@apache.org>.
SilverNarcissus commented on a change in pull request #1851:
URL: https://github.com/apache/iotdb/pull/1851#discussion_r511675529
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/MLogWriter.java
##########
@@ -38,6 +41,8 @@
private static final String STRING_TYPE = "%s,%s,%s";
private File logFile;
private BufferedWriter writer;
Review comment:
We can use file channel to replace it. Check other place where it is used ~
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] wshao08 commented on a change in pull request #1851: [IOTDB-951] Fix auto-creating schema in parallel may write unrecognizable timeseries in mlog.txt
Posted by GitBox <gi...@apache.org>.
wshao08 commented on a change in pull request #1851:
URL: https://github.com/apache/iotdb/pull/1851#discussion_r511759074
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/MLogWriter.java
##########
@@ -38,6 +41,8 @@
private static final String STRING_TYPE = "%s,%s,%s";
private File logFile;
private BufferedWriter writer;
Review comment:
Fixed
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/MLogWriter.java
##########
@@ -38,6 +41,8 @@
private static final String STRING_TYPE = "%s,%s,%s";
private File logFile;
private BufferedWriter writer;
Review comment:
Fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] JackieTien97 commented on a change in pull request #1851: [IOTDB-954] Fix auto-creating schema in parallel may write unrecognizable timeseries in mlog.txt
Posted by GitBox <gi...@apache.org>.
JackieTien97 commented on a change in pull request #1851:
URL: https://github.com/apache/iotdb/pull/1851#discussion_r510868054
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/MLogWriter.java
##########
@@ -38,6 +41,8 @@
private static final String STRING_TYPE = "%s,%s,%s";
private File logFile;
private BufferedWriter writer;
Review comment:
You can use fileChannel to totally replace it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] samperson1997 merged pull request #1851: [IOTDB-951] Fix auto-creating schema in parallel may write unrecognizable timeseries in mlog.txt
Posted by GitBox <gi...@apache.org>.
samperson1997 merged pull request #1851:
URL: https://github.com/apache/iotdb/pull/1851
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org