You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by "shuwenwei (via GitHub)" <gi...@apache.org> on 2026/03/13 10:29:40 UTC

[PR] Fix incorrect measurement schema during compaction [iotdb]

shuwenwei opened a new pull request, #17297:
URL: https://github.com/apache/iotdb/pull/17297

   ## Description
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix incorrect measurement schema during compaction [iotdb]

Posted by "JackieTien97 (via GitHub)" <gi...@apache.org>.
JackieTien97 merged PR #17297:
URL: https://github.com/apache/iotdb/pull/17297


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix incorrect measurement schema during compaction [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17297:
URL: https://github.com/apache/iotdb/pull/17297#issuecomment-4064601093

   ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=17297) **Quality Gate passed**  
   Issues  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [13 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=17297&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=17297&issueStatuses=ACCEPTED)
   
   Measures  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=17297&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=17297&metric=new_coverage&view=list)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=17297&metric=new_duplicated_lines_density&view=list)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=17297)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix incorrect measurement schema during compaction [iotdb]

Posted by "jt2594838 (via GitHub)" <gi...@apache.org>.
jt2594838 commented on code in PR #17297:
URL: https://github.com/apache/iotdb/pull/17297#discussion_r2944671206


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/utils/MultiTsFileDeviceIterator.java:
##########
@@ -347,14 +438,81 @@ public Map<String, CompactionSeriesContext> getCompactionSeriesContextOfCurrentD
    *     endOffset>
    * @throws IOException if io errors occurred
    */
-  @SuppressWarnings({"checkstyle:AtclauseOrderCheck", "squid:S3824"})
   public Map<String, Pair<MeasurementSchema, Map<TsFileResource, Pair<Long, Long>>>>
       getTimeseriesSchemaAndMetadataOffsetOfCurrentDevice() throws IOException {
+    return ignoreAllNullRows
+        ? getTimeseriesSchemaAndMetadataOffsetOfCurrentDeviceForTree()
+        : getTimeseriesSchemaAndMetadataOffsetOfCurrentDeviceForTable();
+  }
+
+  @SuppressWarnings({"checkstyle:AtclauseOrderCheck", "squid:S3824"})
+  public Map<String, Pair<MeasurementSchema, Map<TsFileResource, Pair<Long, Long>>>>
+      getTimeseriesSchemaAndMetadataOffsetOfCurrentDeviceForTree() throws IOException {
     Map<String, Pair<MeasurementSchema, Map<TsFileResource, Pair<Long, Long>>>>
         timeseriesMetadataOffsetMap = new LinkedHashMap<>();
+    Set<MeasurementSchema> seriesNeedToUpdateDataType = new LinkedHashSet<>();
     for (TsFileResource resource : tsFileResourcesSortedByDesc) {
-      if (!deviceIteratorMap.containsKey(resource)
-          || !deviceIteratorMap.get(resource).current().equals(currentDevice)) {
+      TsFileDeviceIterator iterator = deviceIteratorMap.get(resource);
+      if (iterator == null || !iterator.current().equals(currentDevice)) {
+        // if this tsfile has no more device or next device is not equals to the current device,
+        // which means this tsfile does not contain the current device, then skip it.
+        continue;
+      }
+
+      CompactionTsFileReader reader = (CompactionTsFileReader) readerMap.get(resource);
+
+      for (Map.Entry<String, Pair<TimeseriesMetadata, Pair<Long, Long>>> entrySet :

Review Comment:
   entrySet -> entry



##########
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/compaction/alterDataType/CompactionDataTypeAlterTest.java:
##########
@@ -0,0 +1,228 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.storageengine.dataregion.compaction.alterDataType;
+
+import org.apache.iotdb.commons.exception.IllegalPathException;
+import org.apache.iotdb.commons.exception.MetadataException;
+import org.apache.iotdb.commons.path.MeasurementPath;
+import org.apache.iotdb.db.exception.StorageEngineException;
+import org.apache.iotdb.db.storageengine.dataregion.compaction.execute.performer.impl.ReadChunkCompactionPerformer;
+import org.apache.iotdb.db.storageengine.dataregion.compaction.execute.task.InnerSpaceCompactionTask;
+import org.apache.iotdb.db.storageengine.dataregion.modification.ModificationFile;
+import org.apache.iotdb.db.storageengine.dataregion.modification.TreeDeletionEntry;
+import org.apache.iotdb.db.storageengine.dataregion.tsfile.TsFileResource;
+import org.apache.iotdb.db.storageengine.dataregion.utils.TsFileResourceUtils;
+
+import org.apache.tsfile.enums.TSDataType;
+import org.apache.tsfile.exception.write.WriteProcessException;
+import org.apache.tsfile.read.TsFileReader;
+import org.apache.tsfile.read.TsFileSequenceReader;
+import org.apache.tsfile.read.common.Path;
+import org.apache.tsfile.read.common.TimeRange;
+import org.apache.tsfile.read.expression.QueryExpression;
+import org.apache.tsfile.read.query.dataset.QueryDataSet;
+import org.apache.tsfile.write.schema.IMeasurementSchema;
+import org.apache.tsfile.write.schema.MeasurementSchema;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+
+@RunWith(Parameterized.class)
+public class CompactionDataTypeAlterTest extends AbstractCompactionAlterDataTypeTest {
+  private boolean reverse;
+  private String performerType;
+
+  public CompactionDataTypeAlterTest(boolean reverse, String performerType) {
+    this.reverse = reverse;
+    this.performerType = performerType;
+  }
+
+  @Parameterized.Parameters(name = "reverse={0} performerType={1}")
+  public static Collection<Object[]> data() {
+    return Arrays.asList(
+        new Object[][] {
+          {true, "read_chunk"},
+          {false, "read_chunk"},
+          {true, "fast"},
+          {false, "fast"},
+          {true, "read_point"},
+          {false, "read_point"},
+        });
+  }
+
+  @Before
+  public void setUp()
+      throws IOException, WriteProcessException, MetadataException, InterruptedException {
+    super.setUp();
+  }
+
+  @After
+  public void tearDown() throws IOException, StorageEngineException {
+    super.tearDown();
+  }
+
+  @Test
+  public void testCompactNonAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    generateDataTypeNotMatchFilesWithNonAlignedSeries();
+    InnerSpaceCompactionTask task =
+        new InnerSpaceCompactionTask(
+            0, tsFileManager, seqResources, true, getPerformer(performerType), 0);
+    Assert.assertTrue(task.start());
+    TsFileResourceUtils.validateTsFileDataCorrectness(tsFileManager.getTsFileList(true).get(0));
+    Assert.assertEquals(
+        1, ((long) tsFileManager.getTsFileList(true).get(0).getStartTime(device).get()));
+    Assert.assertEquals(
+        2, ((long) tsFileManager.getTsFileList(true).get(0).getEndTime(device).get()));
+    TsFileResource tsFileResource = tsFileManager.getTsFileList(true).get(0);
+    try (TsFileSequenceReader reader =
+            new TsFileSequenceReader(tsFileResource.getTsFile().getAbsolutePath());
+        TsFileReader readTsFile = new TsFileReader(reader)) {
+      ArrayList<Path> paths = new ArrayList<>();
+      paths.add(new Path(device, "s1", true));
+      QueryExpression queryExpression = QueryExpression.create(paths, null);
+      QueryDataSet queryDataSet = readTsFile.query(queryExpression);
+      Assert.assertTrue(queryDataSet.hasNext());
+      Assert.assertEquals("1\t1.0", queryDataSet.next().toString());
+      Assert.assertTrue(queryDataSet.hasNext());
+      Assert.assertEquals("2\t2.0", queryDataSet.next().toString());
+      Assert.assertFalse(queryDataSet.hasNext());
+    }
+  }
+
+  @Test
+  public void testCompactAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    generateDataTypeNotMatchFilesWithAlignedSeries();
+    InnerSpaceCompactionTask task =
+        new InnerSpaceCompactionTask(
+            0, tsFileManager, seqResources, true, getPerformer(performerType), 0);
+    Assert.assertTrue(task.start());
+    TsFileResourceUtils.validateTsFileDataCorrectness(tsFileManager.getTsFileList(true).get(0));
+    Assert.assertEquals(
+        1, ((long) tsFileManager.getTsFileList(true).get(0).getStartTime(device).get()));
+    Assert.assertEquals(
+        2, ((long) tsFileManager.getTsFileList(true).get(0).getEndTime(device).get()));
+    TsFileResource tsFileResource = tsFileManager.getTsFileList(true).get(0);
+    try (TsFileSequenceReader reader =
+            new TsFileSequenceReader(tsFileResource.getTsFile().getAbsolutePath());
+        TsFileReader readTsFile = new TsFileReader(reader)) {
+      ArrayList<Path> paths = new ArrayList<>();
+      paths.add(new Path(device, "s1", true));
+      paths.add(new Path(device, "s2", true));
+      QueryExpression queryExpression = QueryExpression.create(paths, null);
+      QueryDataSet queryDataSet = readTsFile.query(queryExpression);
+      Assert.assertTrue(queryDataSet.hasNext());
+      Assert.assertEquals("1\t1.0\t1.0", queryDataSet.next().toString());
+      Assert.assertTrue(queryDataSet.hasNext());
+      Assert.assertEquals("2\t2.0\t2.0", queryDataSet.next().toString());
+      Assert.assertFalse(queryDataSet.hasNext());
+    }
+  }
+
+  private void generateDataTypeNotMatchFilesWithNonAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    if (!reverse) {
+      TsFileResource resource1 = generateInt32NonAlignedSeriesFile(new TimeRange(1, 1), true);
+      seqResources.add(resource1);
+
+      TsFileResource resource2 = generateDoubleNonAlignedSeriesFile(new TimeRange(2, 2), true);
+      seqResources.add(resource2);
+    } else {
+      TsFileResource resource1 = generateDoubleNonAlignedSeriesFile(new TimeRange(1, 1), true);
+      seqResources.add(resource1);
+
+      TsFileResource resource2 = generateInt32NonAlignedSeriesFile(new TimeRange(2, 2), true);
+      seqResources.add(resource2);
+    }
+
+    schemaFetcher
+        .getSchemaTree()
+        .appendSingleMeasurementPath(
+            new MeasurementPath(device, "s1", new MeasurementSchema("s1", TSDataType.DOUBLE)));

Review Comment:
   Add a case where s1 is still int32?



##########
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/compaction/alterDataType/CompactionDataTypeAlterTest.java:
##########
@@ -0,0 +1,228 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.storageengine.dataregion.compaction.alterDataType;
+
+import org.apache.iotdb.commons.exception.IllegalPathException;
+import org.apache.iotdb.commons.exception.MetadataException;
+import org.apache.iotdb.commons.path.MeasurementPath;
+import org.apache.iotdb.db.exception.StorageEngineException;
+import org.apache.iotdb.db.storageengine.dataregion.compaction.execute.performer.impl.ReadChunkCompactionPerformer;
+import org.apache.iotdb.db.storageengine.dataregion.compaction.execute.task.InnerSpaceCompactionTask;
+import org.apache.iotdb.db.storageengine.dataregion.modification.ModificationFile;
+import org.apache.iotdb.db.storageengine.dataregion.modification.TreeDeletionEntry;
+import org.apache.iotdb.db.storageengine.dataregion.tsfile.TsFileResource;
+import org.apache.iotdb.db.storageengine.dataregion.utils.TsFileResourceUtils;
+
+import org.apache.tsfile.enums.TSDataType;
+import org.apache.tsfile.exception.write.WriteProcessException;
+import org.apache.tsfile.read.TsFileReader;
+import org.apache.tsfile.read.TsFileSequenceReader;
+import org.apache.tsfile.read.common.Path;
+import org.apache.tsfile.read.common.TimeRange;
+import org.apache.tsfile.read.expression.QueryExpression;
+import org.apache.tsfile.read.query.dataset.QueryDataSet;
+import org.apache.tsfile.write.schema.IMeasurementSchema;
+import org.apache.tsfile.write.schema.MeasurementSchema;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+
+@RunWith(Parameterized.class)
+public class CompactionDataTypeAlterTest extends AbstractCompactionAlterDataTypeTest {
+  private boolean reverse;
+  private String performerType;
+
+  public CompactionDataTypeAlterTest(boolean reverse, String performerType) {
+    this.reverse = reverse;
+    this.performerType = performerType;
+  }
+
+  @Parameterized.Parameters(name = "reverse={0} performerType={1}")
+  public static Collection<Object[]> data() {
+    return Arrays.asList(
+        new Object[][] {
+          {true, "read_chunk"},
+          {false, "read_chunk"},
+          {true, "fast"},
+          {false, "fast"},
+          {true, "read_point"},
+          {false, "read_point"},
+        });
+  }
+
+  @Before
+  public void setUp()
+      throws IOException, WriteProcessException, MetadataException, InterruptedException {
+    super.setUp();
+  }
+
+  @After
+  public void tearDown() throws IOException, StorageEngineException {
+    super.tearDown();
+  }
+
+  @Test
+  public void testCompactNonAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    generateDataTypeNotMatchFilesWithNonAlignedSeries();
+    InnerSpaceCompactionTask task =
+        new InnerSpaceCompactionTask(
+            0, tsFileManager, seqResources, true, getPerformer(performerType), 0);
+    Assert.assertTrue(task.start());
+    TsFileResourceUtils.validateTsFileDataCorrectness(tsFileManager.getTsFileList(true).get(0));
+    Assert.assertEquals(
+        1, ((long) tsFileManager.getTsFileList(true).get(0).getStartTime(device).get()));
+    Assert.assertEquals(
+        2, ((long) tsFileManager.getTsFileList(true).get(0).getEndTime(device).get()));
+    TsFileResource tsFileResource = tsFileManager.getTsFileList(true).get(0);
+    try (TsFileSequenceReader reader =
+            new TsFileSequenceReader(tsFileResource.getTsFile().getAbsolutePath());
+        TsFileReader readTsFile = new TsFileReader(reader)) {
+      ArrayList<Path> paths = new ArrayList<>();
+      paths.add(new Path(device, "s1", true));
+      QueryExpression queryExpression = QueryExpression.create(paths, null);
+      QueryDataSet queryDataSet = readTsFile.query(queryExpression);
+      Assert.assertTrue(queryDataSet.hasNext());
+      Assert.assertEquals("1\t1.0", queryDataSet.next().toString());
+      Assert.assertTrue(queryDataSet.hasNext());
+      Assert.assertEquals("2\t2.0", queryDataSet.next().toString());
+      Assert.assertFalse(queryDataSet.hasNext());
+    }
+  }
+
+  @Test
+  public void testCompactAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    generateDataTypeNotMatchFilesWithAlignedSeries();
+    InnerSpaceCompactionTask task =
+        new InnerSpaceCompactionTask(
+            0, tsFileManager, seqResources, true, getPerformer(performerType), 0);
+    Assert.assertTrue(task.start());
+    TsFileResourceUtils.validateTsFileDataCorrectness(tsFileManager.getTsFileList(true).get(0));
+    Assert.assertEquals(
+        1, ((long) tsFileManager.getTsFileList(true).get(0).getStartTime(device).get()));
+    Assert.assertEquals(
+        2, ((long) tsFileManager.getTsFileList(true).get(0).getEndTime(device).get()));
+    TsFileResource tsFileResource = tsFileManager.getTsFileList(true).get(0);
+    try (TsFileSequenceReader reader =
+            new TsFileSequenceReader(tsFileResource.getTsFile().getAbsolutePath());
+        TsFileReader readTsFile = new TsFileReader(reader)) {
+      ArrayList<Path> paths = new ArrayList<>();
+      paths.add(new Path(device, "s1", true));
+      paths.add(new Path(device, "s2", true));
+      QueryExpression queryExpression = QueryExpression.create(paths, null);
+      QueryDataSet queryDataSet = readTsFile.query(queryExpression);
+      Assert.assertTrue(queryDataSet.hasNext());
+      Assert.assertEquals("1\t1.0\t1.0", queryDataSet.next().toString());
+      Assert.assertTrue(queryDataSet.hasNext());
+      Assert.assertEquals("2\t2.0\t2.0", queryDataSet.next().toString());
+      Assert.assertFalse(queryDataSet.hasNext());
+    }
+  }
+
+  private void generateDataTypeNotMatchFilesWithNonAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    if (!reverse) {
+      TsFileResource resource1 = generateInt32NonAlignedSeriesFile(new TimeRange(1, 1), true);
+      seqResources.add(resource1);
+
+      TsFileResource resource2 = generateDoubleNonAlignedSeriesFile(new TimeRange(2, 2), true);
+      seqResources.add(resource2);
+    } else {
+      TsFileResource resource1 = generateDoubleNonAlignedSeriesFile(new TimeRange(1, 1), true);
+      seqResources.add(resource1);
+
+      TsFileResource resource2 = generateInt32NonAlignedSeriesFile(new TimeRange(2, 2), true);
+      seqResources.add(resource2);
+    }
+
+    schemaFetcher
+        .getSchemaTree()
+        .appendSingleMeasurementPath(
+            new MeasurementPath(device, "s1", new MeasurementSchema("s1", TSDataType.DOUBLE)));
+  }
+
+  private void generateDataTypeNotMatchFilesWithAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    if (!reverse) {
+      TsFileResource resource1 = generateInt32AlignedSeriesFile(new TimeRange(1, 1), true);
+      seqResources.add(resource1);
+      TsFileResource resource2 = generateDoubleAlignedSeriesFile(new TimeRange(2, 2), true);
+      seqResources.add(resource2);
+    } else {
+      TsFileResource resource1 = generateDoubleAlignedSeriesFile(new TimeRange(1, 1), true);
+      seqResources.add(resource1);
+      TsFileResource resource2 = generateInt32AlignedSeriesFile(new TimeRange(2, 2), true);
+      seqResources.add(resource2);
+    }
+
+    List<IMeasurementSchema> measurementSchemas2 = new ArrayList<>();
+    measurementSchemas2.add(new MeasurementSchema("s1", TSDataType.DOUBLE));
+    measurementSchemas2.add(new MeasurementSchema("s2", TSDataType.DOUBLE));
+
+    MeasurementPath s1Path = new MeasurementPath(device, "s1", measurementSchemas2.get(0));
+    s1Path.setUnderAlignedEntity(true);
+    MeasurementPath s2Path = new MeasurementPath(device, "s2", measurementSchemas2.get(1));
+    s2Path.setUnderAlignedEntity(true);
+    schemaFetcher.getSchemaTree().appendSingleMeasurementPath(s1Path);
+    schemaFetcher.getSchemaTree().appendSingleMeasurementPath(s2Path);
+  }
+
+  @Test
+  public void testAlterDataTypeWithAlignedSeriesWithTimeDeletion()
+      throws IOException, WriteProcessException, IllegalPathException {
+    List<IMeasurementSchema> measurementSchemas1 = new ArrayList<>();
+    measurementSchemas1.add(new MeasurementSchema("s1", TSDataType.INT32));
+    measurementSchemas1.add(new MeasurementSchema("s2", TSDataType.INT32));
+
+    TsFileResource resource1 = generateInt32AlignedSeriesFile(new TimeRange(0, 100), true);
+    ModificationFile modFile = resource1.getExclusiveModFile();
+    modFile.write(new TreeDeletionEntry(new MeasurementPath(device + ".*"), 0, 40));
+    modFile.close();
+    seqResources.add(resource1);
+
+    List<IMeasurementSchema> measurementSchemas2 = new ArrayList<>();
+    measurementSchemas2.add(new MeasurementSchema("s1", TSDataType.DOUBLE));
+    measurementSchemas2.add(new MeasurementSchema("s2", TSDataType.DOUBLE));
+    TsFileResource resource2 = generateDoubleAlignedSeriesFile(new TimeRange(200, 200), true);
+    seqResources.add(resource2);
+
+    MeasurementPath s1Path = new MeasurementPath(device, "s1", measurementSchemas2.get(0));
+    s1Path.setUnderAlignedEntity(true);
+    MeasurementPath s2Path = new MeasurementPath(device, "s2", measurementSchemas2.get(1));
+    s2Path.setUnderAlignedEntity(true);
+    schemaFetcher.getSchemaTree().appendSingleMeasurementPath(s1Path);
+    schemaFetcher.getSchemaTree().appendSingleMeasurementPath(s2Path);
+
+    InnerSpaceCompactionTask task =
+        new InnerSpaceCompactionTask(
+            0, tsFileManager, seqResources, true, new ReadChunkCompactionPerformer(), 0);
+    Assert.assertTrue(task.start());

Review Comment:
   Check the target file?



##########
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/compaction/alterDataType/CompactionDataTypeNotMatchAlterableDataTypeTest.java:
##########
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.storageengine.dataregion.compaction.alterDataType;
+
+import org.apache.iotdb.commons.exception.IllegalPathException;
+import org.apache.iotdb.commons.exception.MetadataException;
+import org.apache.iotdb.commons.path.MeasurementPath;
+import org.apache.iotdb.db.exception.StorageEngineException;
+import org.apache.iotdb.db.storageengine.dataregion.compaction.execute.task.InnerSpaceCompactionTask;
+import org.apache.iotdb.db.storageengine.dataregion.tsfile.TsFileResource;
+import org.apache.iotdb.db.storageengine.dataregion.tsfile.TsFileResourceStatus;
+import org.apache.iotdb.db.storageengine.dataregion.utils.TsFileResourceUtils;
+
+import org.apache.tsfile.enums.TSDataType;
+import org.apache.tsfile.exception.write.WriteProcessException;
+import org.apache.tsfile.read.common.Path;
+import org.apache.tsfile.read.common.TimeRange;
+import org.apache.tsfile.write.TsFileWriter;
+import org.apache.tsfile.write.record.TSRecord;
+import org.apache.tsfile.write.record.datapoint.DoubleDataPoint;
+import org.apache.tsfile.write.record.datapoint.FloatDataPoint;
+import org.apache.tsfile.write.schema.IMeasurementSchema;
+import org.apache.tsfile.write.schema.MeasurementSchema;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+
+@SuppressWarnings("OptionalGetWithoutIsPresent")
+@RunWith(Parameterized.class)
+public class CompactionDataTypeNotMatchAlterableDataTypeTest
+    extends AbstractCompactionAlterDataTypeTest {
+
+  private String performerType;
+
+  @Before
+  public void setUp()
+      throws IOException, WriteProcessException, MetadataException, InterruptedException {
+    super.setUp();
+  }
+
+  @After
+  public void tearDown() throws IOException, StorageEngineException {
+    super.tearDown();
+  }
+
+  @Parameterized.Parameters(name = "type={0}")
+  public static Collection<Object[]> data() {
+    return Arrays.asList(
+        new Object[][] {
+          {"read_chunk"}, {"fast"}, {"read_point"},
+        });
+  }
+
+  public CompactionDataTypeNotMatchAlterableDataTypeTest(String type) {
+    this.performerType = type;
+  }
+
+  @Test
+  public void testCompactNonAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    generateDataTypeNotMatchFilesWithNonAlignedSeries();
+    InnerSpaceCompactionTask task =
+        new InnerSpaceCompactionTask(
+            0, tsFileManager, seqResources, true, getPerformer(performerType), 0);
+    Assert.assertTrue(task.start());
+    TsFileResourceUtils.validateTsFileDataCorrectness(tsFileManager.getTsFileList(true).get(0));
+    Assert.assertEquals(
+        1, ((long) tsFileManager.getTsFileList(true).get(0).getStartTime(device).get()));

Review Comment:
   Check end time?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix incorrect measurement schema during compaction [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17297:
URL: https://github.com/apache/iotdb/pull/17297#issuecomment-4073034415

   ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=17297) **Quality Gate passed**  
   Issues  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [14 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=17297&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=17297&issueStatuses=ACCEPTED)
   
   Measures  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=17297&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=17297&metric=new_coverage&view=list)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=17297&metric=new_duplicated_lines_density&view=list)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=17297)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix incorrect measurement schema during compaction [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17297:
URL: https://github.com/apache/iotdb/pull/17297#issuecomment-4071802898

   ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=17297) **Quality Gate passed**  
   Issues  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [14 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=17297&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=17297&issueStatuses=ACCEPTED)
   
   Measures  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=17297&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=17297&metric=new_coverage&view=list)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=17297&metric=new_duplicated_lines_density&view=list)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=17297)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix incorrect measurement schema during compaction [iotdb]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #17297:
URL: https://github.com/apache/iotdb/pull/17297#discussion_r2944107160


##########
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/compaction/alterDataType/CompactionDataTypeNotMatchAlterableDataTypeTest.java:
##########
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.storageengine.dataregion.compaction.alterDataType;
+
+import org.apache.iotdb.commons.exception.IllegalPathException;
+import org.apache.iotdb.commons.exception.MetadataException;
+import org.apache.iotdb.commons.path.MeasurementPath;
+import org.apache.iotdb.db.exception.StorageEngineException;
+import org.apache.iotdb.db.storageengine.dataregion.compaction.execute.task.InnerSpaceCompactionTask;
+import org.apache.iotdb.db.storageengine.dataregion.tsfile.TsFileResource;
+import org.apache.iotdb.db.storageengine.dataregion.tsfile.TsFileResourceStatus;
+import org.apache.iotdb.db.storageengine.dataregion.utils.TsFileResourceUtils;
+
+import org.apache.tsfile.enums.TSDataType;
+import org.apache.tsfile.exception.write.WriteProcessException;
+import org.apache.tsfile.read.common.Path;
+import org.apache.tsfile.read.common.TimeRange;
+import org.apache.tsfile.write.TsFileWriter;
+import org.apache.tsfile.write.record.TSRecord;
+import org.apache.tsfile.write.record.datapoint.DoubleDataPoint;
+import org.apache.tsfile.write.record.datapoint.FloatDataPoint;
+import org.apache.tsfile.write.schema.IMeasurementSchema;
+import org.apache.tsfile.write.schema.MeasurementSchema;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+
+@SuppressWarnings("OptionalGetWithoutIsPresent")
+@RunWith(Parameterized.class)
+public class CompactionDataTypeNotMatchAlterableDataTypeTest
+    extends AbstractCompactionAlterDataTypeTest {
+
+  private String performerType;
+
+  @Before
+  public void setUp()
+      throws IOException, WriteProcessException, MetadataException, InterruptedException {
+    super.setUp();
+  }
+
+  @After
+  public void tearDown() throws IOException, StorageEngineException {
+    super.tearDown();
+  }
+
+  @Parameterized.Parameters(name = "type={0}")
+  public static Collection<Object[]> data() {
+    return Arrays.asList(
+        new Object[][] {
+          {"read_chunk"}, {"fast"}, {"read_point"},
+        });
+  }
+
+  public CompactionDataTypeNotMatchAlterableDataTypeTest(String type) {
+    this.performerType = type;
+  }
+
+  @Test
+  public void testCompactNonAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    generateDataTypeNotMatchFilesWithNonAlignedSeries();
+    InnerSpaceCompactionTask task =
+        new InnerSpaceCompactionTask(
+            0, tsFileManager, seqResources, true, getPerformer(performerType), 0);
+    Assert.assertTrue(task.start());
+    TsFileResourceUtils.validateTsFileDataCorrectness(tsFileManager.getTsFileList(true).get(0));
+    Assert.assertEquals(
+        1, ((long) tsFileManager.getTsFileList(true).get(0).getStartTime(device).get()));
+  }
+
+  @Test
+  public void testCompactAlignedSeriesWithReadChunkCompactionPerformer()
+      throws IOException, WriteProcessException, IllegalPathException {
+    generateDataTypeNotMatchFilesWithAlignedSeries();
+    InnerSpaceCompactionTask task =
+        new InnerSpaceCompactionTask(
+            0, tsFileManager, seqResources, true, getPerformer(performerType), 0);
+    Assert.assertTrue(task.start());
+    TsFileResourceUtils.validateTsFileDataCorrectness(tsFileManager.getTsFileList(true).get(0));
+    Assert.assertEquals(
+        1, ((long) tsFileManager.getTsFileList(true).get(0).getStartTime(device).get()));
+  }
+
+  private void generateDataTypeNotMatchFilesWithNonAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    MeasurementSchema measurementSchema1 = new MeasurementSchema("s1", TSDataType.INT32);
+    TsFileResource resource1 = generateInt32NonAlignedSeriesFile(new TimeRange(1, 1), true);
+    seqResources.add(resource1);
+
+    MeasurementSchema measurementSchema2 = new MeasurementSchema("s1", TSDataType.FLOAT);
+    TsFileResource resource2 = generateFloatNonAlignedSeriesFile(new TimeRange(2, 2), true);
+    seqResources.add(resource2);
+
+    schemaFetcher
+        .getSchemaTree()
+        .appendSingleMeasurementPath(new MeasurementPath(device, "s1", measurementSchema2));
+  }
+
+  private void generateDataTypeNotMatchFilesWithAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    List<IMeasurementSchema> measurementSchemas1 = new ArrayList<>();
+    measurementSchemas1.add(new MeasurementSchema("s1", TSDataType.INT32));
+    measurementSchemas1.add(new MeasurementSchema("s2", TSDataType.INT32));
+

Review Comment:
   `measurementSchemas1` is built but never used (the test uses `generateInt32AlignedSeriesFile(...)` which defines its own schemas). Removing this unused local variable will avoid confusion about which schema set is intended to drive the file generation.
   



##########
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/compaction/alterDataType/CompactionDataTypeNotMatchAlterableDataTypeTest.java:
##########
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.storageengine.dataregion.compaction.alterDataType;
+
+import org.apache.iotdb.commons.exception.IllegalPathException;
+import org.apache.iotdb.commons.exception.MetadataException;
+import org.apache.iotdb.commons.path.MeasurementPath;
+import org.apache.iotdb.db.exception.StorageEngineException;
+import org.apache.iotdb.db.storageengine.dataregion.compaction.execute.task.InnerSpaceCompactionTask;
+import org.apache.iotdb.db.storageengine.dataregion.tsfile.TsFileResource;
+import org.apache.iotdb.db.storageengine.dataregion.tsfile.TsFileResourceStatus;
+import org.apache.iotdb.db.storageengine.dataregion.utils.TsFileResourceUtils;
+
+import org.apache.tsfile.enums.TSDataType;
+import org.apache.tsfile.exception.write.WriteProcessException;
+import org.apache.tsfile.read.common.Path;
+import org.apache.tsfile.read.common.TimeRange;
+import org.apache.tsfile.write.TsFileWriter;
+import org.apache.tsfile.write.record.TSRecord;
+import org.apache.tsfile.write.record.datapoint.DoubleDataPoint;
+import org.apache.tsfile.write.record.datapoint.FloatDataPoint;
+import org.apache.tsfile.write.schema.IMeasurementSchema;
+import org.apache.tsfile.write.schema.MeasurementSchema;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+
+@SuppressWarnings("OptionalGetWithoutIsPresent")
+@RunWith(Parameterized.class)
+public class CompactionDataTypeNotMatchAlterableDataTypeTest
+    extends AbstractCompactionAlterDataTypeTest {
+
+  private String performerType;
+
+  @Before
+  public void setUp()
+      throws IOException, WriteProcessException, MetadataException, InterruptedException {
+    super.setUp();
+  }
+
+  @After
+  public void tearDown() throws IOException, StorageEngineException {
+    super.tearDown();
+  }
+
+  @Parameterized.Parameters(name = "type={0}")
+  public static Collection<Object[]> data() {
+    return Arrays.asList(
+        new Object[][] {
+          {"read_chunk"}, {"fast"}, {"read_point"},
+        });
+  }
+
+  public CompactionDataTypeNotMatchAlterableDataTypeTest(String type) {
+    this.performerType = type;
+  }
+
+  @Test
+  public void testCompactNonAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    generateDataTypeNotMatchFilesWithNonAlignedSeries();
+    InnerSpaceCompactionTask task =
+        new InnerSpaceCompactionTask(
+            0, tsFileManager, seqResources, true, getPerformer(performerType), 0);
+    Assert.assertTrue(task.start());
+    TsFileResourceUtils.validateTsFileDataCorrectness(tsFileManager.getTsFileList(true).get(0));
+    Assert.assertEquals(
+        1, ((long) tsFileManager.getTsFileList(true).get(0).getStartTime(device).get()));
+  }
+
+  @Test
+  public void testCompactAlignedSeriesWithReadChunkCompactionPerformer()

Review Comment:
   `testCompactAlignedSeriesWithReadChunkCompactionPerformer` is parameterized and runs for `read_chunk`, `fast`, and `read_point`, so the method name is misleading. Rename it to a performer-agnostic name (e.g., `testCompactAlignedSeries`) to reflect what it actually validates and to keep test reports accurate.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix incorrect measurement schema during compaction [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17297:
URL: https://github.com/apache/iotdb/pull/17297#issuecomment-4071942037

   ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=17297) **Quality Gate passed**  
   Issues  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [14 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=17297&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=17297&issueStatuses=ACCEPTED)
   
   Measures  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=17297&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=17297&metric=new_coverage&view=list)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=17297&metric=new_duplicated_lines_density&view=list)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=17297)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix incorrect measurement schema during compaction [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17297:
URL: https://github.com/apache/iotdb/pull/17297#issuecomment-4072738743

   ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=17297) **Quality Gate passed**  
   Issues  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [14 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=17297&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=17297&issueStatuses=ACCEPTED)
   
   Measures  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=17297&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=17297&metric=new_coverage&view=list)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=17297&metric=new_duplicated_lines_density&view=list)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=17297)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix incorrect measurement schema during compaction [iotdb]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #17297:
URL: https://github.com/apache/iotdb/pull/17297#discussion_r2930378907


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/utils/MultiTsFileDeviceIterator.java:
##########
@@ -275,16 +291,75 @@ public Map<String, MeasurementSchema> getAllSchemasOfCurrentDevice() throws IOEx
       reader.getDeviceTimeseriesMetadata(
           timeseriesMetadataList,
           deviceIteratorMap.get(resource).getFirstMeasurementNodeOfCurrentDevice(),
+          seriesNeedToUpdateDataType,
+          true,
+          null);
+      for (TimeseriesMetadata timeseriesMetadata : timeseriesMetadataList) {
+        MeasurementSchema measurementSchema = schemaMap.get(timeseriesMetadata.getMeasurementId());
+        if (measurementSchema == null) {
+          if (!timeseriesMetadata.getChunkMetadataList().isEmpty()) {
+            schemaMap.put(
+                timeseriesMetadata.getMeasurementId(),
+                reader.getMeasurementSchema(timeseriesMetadata.getChunkMetadataList()));
+          }
+          continue;
+        }
+        if (measurementSchema.getType() != timeseriesMetadata.getTsDataType()) {
+          seriesNeedToUpdateDataType.add(timeseriesMetadata.getMeasurementId());
+        }
+      }
+    }
+    List<IMeasurementSchema> latestMeasurementSchemas =
+        CompactionUtils.getLatestMeasurementSchemasForTreeModel(
+            currentDevice.left, new ArrayList<>(seriesNeedToUpdateDataType));
+    for (IMeasurementSchema latestMeasurementSchema : latestMeasurementSchemas) {
+      if (latestMeasurementSchema != null) {
+        schemaMap.put(
+            latestMeasurementSchema.getMeasurementName(),
+            (MeasurementSchema) latestMeasurementSchema);
+      }
+    }
+    return schemaMap;
+  }
+
+  private Map<String, MeasurementSchema> getAllSchemasOfCurrentDeviceForTable() throws IOException {
+    Map<String, MeasurementSchema> schemaMap = new ConcurrentHashMap<>();
+    TsTable tsTable =
+        DataNodeTableCache.getInstance().getTable(databaseName, currentDevice.left.getTableName());
+    // get schemas from the newest file to the oldest file
+    for (TsFileResource resource : tsFileResourcesSortedByDesc) {
+      TsFileDeviceIterator deviceIterator = deviceIteratorMap.get(resource);
+      if (!deviceIteratorMap.containsKey(resource)
+          || !deviceIterator.current().equals(currentDevice)) {
+        // if this tsfile has no more device or next device is not equals to the current device,
+        // which means this tsfile does not contain the current device, then skip it.
+        continue;
+      }
+      TsFileSequenceReader reader = readerMap.get(resource);
+      List<TimeseriesMetadata> timeseriesMetadataList = new ArrayList<>();
+      reader.getDeviceTimeseriesMetadata(
+          timeseriesMetadataList,
+          deviceIterator.getFirstMeasurementNodeOfCurrentDevice(),
           schemaMap.keySet(),
           true,
           null);
       for (TimeseriesMetadata timeseriesMetadata : timeseriesMetadataList) {
-        if (!schemaMap.containsKey(timeseriesMetadata.getMeasurementId())
-            && !timeseriesMetadata.getChunkMetadataList().isEmpty()) {
-          schemaMap.put(
-              timeseriesMetadata.getMeasurementId(),
-              reader.getMeasurementSchema(timeseriesMetadata.getChunkMetadataList()));
+        MeasurementSchema measurementSchema = schemaMap.get(timeseriesMetadata.getMeasurementId());
+        if (measurementSchema != null) {
+          continue;
+        }
+        if (tsTable != null) {
+          TsTableColumnSchema columnSchema =
+              tsTable.getColumnSchema(timeseriesMetadata.getMeasurementId());
+          if (columnSchema != null) {
+            measurementSchema = (MeasurementSchema) columnSchema.getMeasurementSchema();
+          }
+        }
+        if (measurementSchema == null && !timeseriesMetadata.getChunkMetadataList().isEmpty()) {
+          measurementSchema =
+              reader.getMeasurementSchema(timeseriesMetadata.getChunkMetadataList());
         }

Review Comment:
   `getAllSchemasOfCurrentDeviceForTable` can insert a null value into `schemaMap` (when `tsTable` has no column schema and `timeseriesMetadata.getChunkMetadataList()` is empty). Since `schemaMap.keySet()` is passed into `getDeviceTimeseriesMetadata(...)` as the exclusion set, adding the key with a null value can prevent older files from being scanned to resolve the schema later, leaving the schema permanently null. Avoid putting the measurement into `schemaMap` until a non-null `MeasurementSchema` is derived, or maintain a separate exclusion set that only contains measurements with resolved schemas.
   



##########
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/compaction/alterDataType/AbstractCompactionAlterDataTypeTest.java:
##########
@@ -0,0 +1,181 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.storageengine.dataregion.compaction.alterDataType;
+
+import org.apache.iotdb.commons.exception.MetadataException;
+import org.apache.iotdb.db.exception.StorageEngineException;
+import org.apache.iotdb.db.storageengine.dataregion.compaction.AbstractCompactionTest;
+import org.apache.iotdb.db.storageengine.dataregion.compaction.execute.utils.CompactionUtils;
+import org.apache.iotdb.db.storageengine.dataregion.compaction.utils.CompactionFakeSchemaFetcherImpl;
+import org.apache.iotdb.db.storageengine.dataregion.tsfile.TsFileResource;
+import org.apache.iotdb.db.storageengine.dataregion.tsfile.TsFileResourceStatus;
+
+import org.apache.tsfile.enums.TSDataType;
+import org.apache.tsfile.exception.write.WriteProcessException;
+import org.apache.tsfile.file.metadata.IDeviceID;
+import org.apache.tsfile.read.common.Path;
+import org.apache.tsfile.read.common.TimeRange;
+import org.apache.tsfile.write.TsFileWriter;
+import org.apache.tsfile.write.record.TSRecord;
+import org.apache.tsfile.write.record.datapoint.DoubleDataPoint;
+import org.apache.tsfile.write.record.datapoint.FloatDataPoint;
+import org.apache.tsfile.write.record.datapoint.IntDataPoint;
+import org.apache.tsfile.write.schema.IMeasurementSchema;
+import org.apache.tsfile.write.schema.MeasurementSchema;
+import org.junit.After;
+import org.junit.Before;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+public class AbstractCompactionAlterDataTypeTest extends AbstractCompactionTest {
+
+  protected final String oldThreadName = Thread.currentThread().getName();
+  protected final IDeviceID device =
+      IDeviceID.Factory.DEFAULT_FACTORY.create(COMPACTION_TEST_SG + ".d1");
+
+  protected CompactionFakeSchemaFetcherImpl schemaFetcher;
+
+  @Before
+  public void setUp()
+      throws IOException, WriteProcessException, MetadataException, InterruptedException {
+    super.setUp();
+    Thread.currentThread().setName("pool-1-IoTDB-Compaction-Worker-1");
+    this.schemaFetcher = new CompactionFakeSchemaFetcherImpl();
+    schemaFetcher.getSchemaTree().setDatabases(Collections.singleton(COMPACTION_TEST_SG));
+    CompactionUtils.setSchemaFetcher(schemaFetcher);
+  }
+
+  @After
+  public void tearDown() throws IOException, StorageEngineException {

Review Comment:
   `setUp()` installs a static/global schema fetcher via `CompactionUtils.setSchemaFetcher(schemaFetcher)`, but `tearDown()` never resets it. Since this is global mutable state, it can leak into other tests executed in the same JVM and change their behavior. Reset it in `tearDown()` (e.g., set it back to null) to keep tests isolated.
   



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/utils/executor/readchunk/ReadChunkAlignedSeriesCompactionExecutor.java:
##########
@@ -133,8 +144,66 @@ public ReadChunkAlignedSeriesCompactionExecutor(
     this.ignoreAllNullRows = ignoreAllNullRows;
   }
 
-  private void collectValueColumnSchemaList() throws IOException {
+  private void collectValueColumnSchemaListForTable(String database) throws IOException {
+    TSFileConfig tsFileConfig = TSFileDescriptor.getInstance().getConfig();
+    TsTable tsTable = DataNodeTableCache.getInstance().getTable(database, device.getTableName());
     Map<String, IMeasurementSchema> measurementSchemaMap = new HashMap<>();
+    for (int i = this.readerAndChunkMetadataList.size() - 1; i >= 0; i--) {
+      Pair<TsFileSequenceReader, List<AbstractAlignedChunkMetadata>> pair =
+          this.readerAndChunkMetadataList.get(i);
+      CompactionTsFileReader reader = (CompactionTsFileReader) pair.getLeft();
+      List<AbstractAlignedChunkMetadata> alignedChunkMetadataList = pair.getRight();
+      for (AbstractAlignedChunkMetadata alignedChunkMetadata : alignedChunkMetadataList) {
+        if (alignedChunkMetadata == null) {
+          continue;
+        }
+        if (timeSchema == null) {
+          timeSchema =
+              new MeasurementSchema(
+                  "",
+                  alignedChunkMetadata.getTimeChunkMetadata().getDataType(),
+                  TSEncoding.valueOf(tsFileConfig.getTimeEncoder()),
+                  tsFileConfig.getCompressor());
+        }
+        for (IChunkMetadata chunkMetadata : alignedChunkMetadata.getValueChunkMetadataList()) {
+          if (chunkMetadata == null
+              || measurementSchemaMap.containsKey(chunkMetadata.getMeasurementUid())) {
+            continue;
+          }
+          TsTableColumnSchema schemaInTsTable =
+              tsTable.getColumnSchema(chunkMetadata.getMeasurementUid());

Review Comment:
   `collectValueColumnSchemaListForTable` dereferences `tsTable` without a null-check (`tsTable.getColumnSchema(...)`). `DataNodeTableCache.getInstance().getTable(...)` can return null (e.g., table not in cache / dropped), which would cause an NPE during compaction. Please guard `tsTable` (treat missing table schema as `schemaInTsTable == null`) and fall back to reading the chunk header schema in that case.
   



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/performer/impl/ReadChunkCompactionPerformer.java:
##########
@@ -304,42 +313,63 @@ private void compactNotAlignedSeries(
 
   private LinkedList<Pair<TsFileSequenceReader, List<ChunkMetadata>>>
       filterDataTypeNotMatchedChunkMetadata(
+          IDeviceID deviceID,
+          String measurement,
           LinkedList<Pair<TsFileSequenceReader, List<ChunkMetadata>>> readerAndChunkMetadataList) {
     if (readerAndChunkMetadataList.isEmpty()) {
       return readerAndChunkMetadataList;
     }
-    LinkedList<Pair<TsFileSequenceReader, List<ChunkMetadata>>> result = new LinkedList<>();
     // find correct data type
     TSDataType correctDataType = null;
-    for (int i = readerAndChunkMetadataList.size() - 1; i >= 0 && correctDataType == null; i--) {
-      List<ChunkMetadata> chunkMetadataList = readerAndChunkMetadataList.get(i).getRight();
-      if (chunkMetadataList == null || chunkMetadataList.isEmpty()) {
-        continue;
-      }
+    boolean hasDifferentDataTypes = false;
+    Iterator<Pair<TsFileSequenceReader, List<ChunkMetadata>>> descIterator =
+        readerAndChunkMetadataList.descendingIterator();
+    while (descIterator.hasNext()) {
+      Pair<TsFileSequenceReader, List<ChunkMetadata>> pair = descIterator.next();
+      List<ChunkMetadata> chunkMetadataList = pair.right;
+      TSDataType dataTypeInCurrentFile = null;
       for (ChunkMetadata chunkMetadata : chunkMetadataList) {
-        if (chunkMetadata == null) {
-          continue;
+        if (chunkMetadata != null) {
+          dataTypeInCurrentFile = chunkMetadata.getDataType();
+          break;
         }
-        correctDataType = chunkMetadata.getDataType();
+      }
+      if (correctDataType == null) {
+        correctDataType = dataTypeInCurrentFile;
+      } else if (correctDataType != dataTypeInCurrentFile) {
+        hasDifferentDataTypes = true;
         break;
       }
     }
-    if (correctDataType == null) {
+    if (!hasDifferentDataTypes) {
       return readerAndChunkMetadataList;
     }
+
+    IMeasurementSchema schema =
+        CompactionUtils.getLatestMeasurementSchemasForTreeModel(
+                deviceID, Collections.singletonList(measurement))
+            .get(0);
+    if (schema != null) {
+      correctDataType = schema.getType();
+    }
+
+    LinkedList<Pair<TsFileSequenceReader, List<ChunkMetadata>>> result = new LinkedList<>();
     // check data type consistent and skip compact files with wrong data type
     for (Pair<TsFileSequenceReader, List<ChunkMetadata>> tsFileSequenceReaderListPair :
         readerAndChunkMetadataList) {
       boolean dataTypeConsistent = true;
       for (ChunkMetadata chunkMetadata : tsFileSequenceReaderListPair.getRight()) {
-        if (chunkMetadata != null
-            && !MetadataUtils.canAlter(chunkMetadata.getDataType(), correctDataType)) {
-          dataTypeConsistent = false;
+        if (chunkMetadata == null) {
+          continue;
+        }
+        if (chunkMetadata.getDataType() == correctDataType) {
           break;
         }
-        if (chunkMetadata != null && chunkMetadata.getDataType() != correctDataType) {
-          chunkMetadata.setNewType(correctDataType);
+        if (!MetadataUtils.canAlter(chunkMetadata.getDataType(), correctDataType)) {
+          dataTypeConsistent = false;
+          break;
         }
+        chunkMetadata.setNewType(correctDataType);

Review Comment:
   In `filterDataTypeNotMatchedChunkMetadata`, `correctDataType` can remain null if the newest file(s) for this series have empty chunk-metadata lists (e.g., fully deleted by modifications), and the schema fetch returns null. In that case `MetadataUtils.canAlter(chunkMetadata.getDataType(), correctDataType)` will throw an NPE. Add a guard such as: if `correctDataType` is still null after scanning / schema lookup, return the original list (or filter out empty pairs) without attempting type alteration.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix incorrect measurement schema during compaction [iotdb]

Posted by "shuwenwei (via GitHub)" <gi...@apache.org>.
shuwenwei commented on code in PR #17297:
URL: https://github.com/apache/iotdb/pull/17297#discussion_r2944963457


##########
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/compaction/alterDataType/CompactionDataTypeAlterTest.java:
##########
@@ -0,0 +1,228 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.storageengine.dataregion.compaction.alterDataType;
+
+import org.apache.iotdb.commons.exception.IllegalPathException;
+import org.apache.iotdb.commons.exception.MetadataException;
+import org.apache.iotdb.commons.path.MeasurementPath;
+import org.apache.iotdb.db.exception.StorageEngineException;
+import org.apache.iotdb.db.storageengine.dataregion.compaction.execute.performer.impl.ReadChunkCompactionPerformer;
+import org.apache.iotdb.db.storageengine.dataregion.compaction.execute.task.InnerSpaceCompactionTask;
+import org.apache.iotdb.db.storageengine.dataregion.modification.ModificationFile;
+import org.apache.iotdb.db.storageengine.dataregion.modification.TreeDeletionEntry;
+import org.apache.iotdb.db.storageengine.dataregion.tsfile.TsFileResource;
+import org.apache.iotdb.db.storageengine.dataregion.utils.TsFileResourceUtils;
+
+import org.apache.tsfile.enums.TSDataType;
+import org.apache.tsfile.exception.write.WriteProcessException;
+import org.apache.tsfile.read.TsFileReader;
+import org.apache.tsfile.read.TsFileSequenceReader;
+import org.apache.tsfile.read.common.Path;
+import org.apache.tsfile.read.common.TimeRange;
+import org.apache.tsfile.read.expression.QueryExpression;
+import org.apache.tsfile.read.query.dataset.QueryDataSet;
+import org.apache.tsfile.write.schema.IMeasurementSchema;
+import org.apache.tsfile.write.schema.MeasurementSchema;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+
+@RunWith(Parameterized.class)
+public class CompactionDataTypeAlterTest extends AbstractCompactionAlterDataTypeTest {
+  private boolean reverse;
+  private String performerType;
+
+  public CompactionDataTypeAlterTest(boolean reverse, String performerType) {
+    this.reverse = reverse;
+    this.performerType = performerType;
+  }
+
+  @Parameterized.Parameters(name = "reverse={0} performerType={1}")
+  public static Collection<Object[]> data() {
+    return Arrays.asList(
+        new Object[][] {
+          {true, "read_chunk"},
+          {false, "read_chunk"},
+          {true, "fast"},
+          {false, "fast"},
+          {true, "read_point"},
+          {false, "read_point"},
+        });
+  }
+
+  @Before
+  public void setUp()
+      throws IOException, WriteProcessException, MetadataException, InterruptedException {
+    super.setUp();
+  }
+
+  @After
+  public void tearDown() throws IOException, StorageEngineException {
+    super.tearDown();
+  }
+
+  @Test
+  public void testCompactNonAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    generateDataTypeNotMatchFilesWithNonAlignedSeries();
+    InnerSpaceCompactionTask task =
+        new InnerSpaceCompactionTask(
+            0, tsFileManager, seqResources, true, getPerformer(performerType), 0);
+    Assert.assertTrue(task.start());
+    TsFileResourceUtils.validateTsFileDataCorrectness(tsFileManager.getTsFileList(true).get(0));
+    Assert.assertEquals(
+        1, ((long) tsFileManager.getTsFileList(true).get(0).getStartTime(device).get()));
+    Assert.assertEquals(
+        2, ((long) tsFileManager.getTsFileList(true).get(0).getEndTime(device).get()));
+    TsFileResource tsFileResource = tsFileManager.getTsFileList(true).get(0);
+    try (TsFileSequenceReader reader =
+            new TsFileSequenceReader(tsFileResource.getTsFile().getAbsolutePath());
+        TsFileReader readTsFile = new TsFileReader(reader)) {
+      ArrayList<Path> paths = new ArrayList<>();
+      paths.add(new Path(device, "s1", true));
+      QueryExpression queryExpression = QueryExpression.create(paths, null);
+      QueryDataSet queryDataSet = readTsFile.query(queryExpression);
+      Assert.assertTrue(queryDataSet.hasNext());
+      Assert.assertEquals("1\t1.0", queryDataSet.next().toString());
+      Assert.assertTrue(queryDataSet.hasNext());
+      Assert.assertEquals("2\t2.0", queryDataSet.next().toString());
+      Assert.assertFalse(queryDataSet.hasNext());
+    }
+  }
+
+  @Test
+  public void testCompactAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    generateDataTypeNotMatchFilesWithAlignedSeries();
+    InnerSpaceCompactionTask task =
+        new InnerSpaceCompactionTask(
+            0, tsFileManager, seqResources, true, getPerformer(performerType), 0);
+    Assert.assertTrue(task.start());
+    TsFileResourceUtils.validateTsFileDataCorrectness(tsFileManager.getTsFileList(true).get(0));
+    Assert.assertEquals(
+        1, ((long) tsFileManager.getTsFileList(true).get(0).getStartTime(device).get()));
+    Assert.assertEquals(
+        2, ((long) tsFileManager.getTsFileList(true).get(0).getEndTime(device).get()));
+    TsFileResource tsFileResource = tsFileManager.getTsFileList(true).get(0);
+    try (TsFileSequenceReader reader =
+            new TsFileSequenceReader(tsFileResource.getTsFile().getAbsolutePath());
+        TsFileReader readTsFile = new TsFileReader(reader)) {
+      ArrayList<Path> paths = new ArrayList<>();
+      paths.add(new Path(device, "s1", true));
+      paths.add(new Path(device, "s2", true));
+      QueryExpression queryExpression = QueryExpression.create(paths, null);
+      QueryDataSet queryDataSet = readTsFile.query(queryExpression);
+      Assert.assertTrue(queryDataSet.hasNext());
+      Assert.assertEquals("1\t1.0\t1.0", queryDataSet.next().toString());
+      Assert.assertTrue(queryDataSet.hasNext());
+      Assert.assertEquals("2\t2.0\t2.0", queryDataSet.next().toString());
+      Assert.assertFalse(queryDataSet.hasNext());
+    }
+  }
+
+  private void generateDataTypeNotMatchFilesWithNonAlignedSeries()
+      throws IOException, WriteProcessException, IllegalPathException {
+    if (!reverse) {
+      TsFileResource resource1 = generateInt32NonAlignedSeriesFile(new TimeRange(1, 1), true);
+      seqResources.add(resource1);
+
+      TsFileResource resource2 = generateDoubleNonAlignedSeriesFile(new TimeRange(2, 2), true);
+      seqResources.add(resource2);
+    } else {
+      TsFileResource resource1 = generateDoubleNonAlignedSeriesFile(new TimeRange(1, 1), true);
+      seqResources.add(resource1);
+
+      TsFileResource resource2 = generateInt32NonAlignedSeriesFile(new TimeRange(2, 2), true);
+      seqResources.add(resource2);
+    }
+
+    schemaFetcher
+        .getSchemaTree()
+        .appendSingleMeasurementPath(
+            new MeasurementPath(device, "s1", new MeasurementSchema("s1", TSDataType.DOUBLE)));

Review Comment:
   Added in org/apache/iotdb/db/storageengine/dataregion/compaction/alterDataType/CompactionDataTypeNotMatchTest.java



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix incorrect measurement schema during compaction [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17297:
URL: https://github.com/apache/iotdb/pull/17297#issuecomment-4065003400

   ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=17297) **Quality Gate passed**  
   Issues  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [14 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=17297&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=17297&issueStatuses=ACCEPTED)
   
   Measures  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=17297&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=17297&metric=new_coverage&view=list)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=17297&metric=new_duplicated_lines_density&view=list)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=17297)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix incorrect measurement schema during compaction [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17297:
URL: https://github.com/apache/iotdb/pull/17297#issuecomment-4071826439

   ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=17297) **Quality Gate passed**  
   Issues  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [14 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=17297&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=17297&issueStatuses=ACCEPTED)
   
   Measures  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=17297&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=17297&metric=new_coverage&view=list)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=17297&metric=new_duplicated_lines_density&view=list)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=17297)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix incorrect measurement schema during compaction [iotdb]

Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #17297:
URL: https://github.com/apache/iotdb/pull/17297#issuecomment-4072274123

   ## [Codecov](https://app.codecov.io/gh/apache/iotdb/pull/17297?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `95.87629%` with `12 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 39.78%. Comparing base ([`e7f8097`](https://app.codecov.io/gh/apache/iotdb/commit/e7f80974364329ea908e04fd630ffd43c145d6de?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`bd712ac`](https://app.codecov.io/gh/apache/iotdb/commit/bd712ac6a66076527bd7a704226eb94a0c74ad96?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 10 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/iotdb/pull/17297?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [...ction/execute/utils/MultiTsFileDeviceIterator.java](https://app.codecov.io/gh/apache/iotdb/pull/17297?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fstorageengine%2Fdataregion%2Fcompaction%2Fexecute%2Futils%2FMultiTsFileDeviceIterator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvc3RvcmFnZWVuZ2luZS9kYXRhcmVnaW9uL2NvbXBhY3Rpb24vZXhlY3V0ZS91dGlscy9NdWx0aVRzRmlsZURldmljZUl0ZXJhdG9yLmphdmE=) | 97.12% | [4 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/17297?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...gion/compaction/execute/utils/CompactionUtils.java](https://app.codecov.io/gh/apache/iotdb/pull/17297?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fstorageengine%2Fdataregion%2Fcompaction%2Fexecute%2Futils%2FCompactionUtils.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvc3RvcmFnZWVuZ2luZS9kYXRhcmVnaW9uL2NvbXBhY3Rpb24vZXhlY3V0ZS91dGlscy9Db21wYWN0aW9uVXRpbHMuamF2YQ==) | 85.00% | [3 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/17297?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...e/performer/impl/ReadChunkCompactionPerformer.java](https://app.codecov.io/gh/apache/iotdb/pull/17297?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fstorageengine%2Fdataregion%2Fcompaction%2Fexecute%2Fperformer%2Fimpl%2FReadChunkCompactionPerformer.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvc3RvcmFnZWVuZ2luZS9kYXRhcmVnaW9uL2NvbXBhY3Rpb24vZXhlY3V0ZS9wZXJmb3JtZXIvaW1wbC9SZWFkQ2h1bmtDb21wYWN0aW9uUGVyZm9ybWVyLmphdmE=) | 94.28% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/17297?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...hunk/ReadChunkAlignedSeriesCompactionExecutor.java](https://app.codecov.io/gh/apache/iotdb/pull/17297?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fstorageengine%2Fdataregion%2Fcompaction%2Fexecute%2Futils%2Fexecutor%2Freadchunk%2FReadChunkAlignedSeriesCompactionExecutor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvc3RvcmFnZWVuZ2luZS9kYXRhcmVnaW9uL2NvbXBhY3Rpb24vZXhlY3V0ZS91dGlscy9leGVjdXRvci9yZWFkY2h1bmsvUmVhZENodW5rQWxpZ25lZFNlcmllc0NvbXBhY3Rpb25FeGVjdXRvci5qYXZh) | 97.53% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/17297?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...org/apache/iotdb/commons/schema/table/TsTable.java](https://app.codecov.io/gh/apache/iotdb/pull/17297?src=pr&el=tree&filepath=iotdb-core%2Fnode-commons%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fcommons%2Fschema%2Ftable%2FTsTable.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9ub2RlLWNvbW1vbnMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2lvdGRiL2NvbW1vbnMvc2NoZW1hL3RhYmxlL1RzVGFibGUuamF2YQ==) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/17297?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #17297      +/-   ##
   ============================================
   + Coverage     39.72%   39.78%   +0.06%     
     Complexity      282      282              
   ============================================
     Files          5101     5099       -2     
     Lines        342143   342513     +370     
     Branches      43583    43660      +77     
   ============================================
   + Hits         135914   136280     +366     
   - Misses       206229   206233       +4     
   ```
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/iotdb/pull/17297?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org