You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2021/04/11 15:21:53 UTC
[GitHub] [iotdb] samperson1997 opened a new pull request #2992: [To Vector] Fix Tablet.addValue()
samperson1997 opened a new pull request #2992:
URL: https://github.com/apache/iotdb/pull/2992
Fix `Tablet.addValue()` and add another method to insertTablet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] samperson1997 commented on a change in pull request #2992: [To Vector] Fix Tablet.addValue()
Posted by GitBox <gi...@apache.org>.
samperson1997 commented on a change in pull request #2992:
URL: https://github.com/apache/iotdb/pull/2992#discussion_r612522499
##########
File path: session/src/main/java/org/apache/iotdb/session/Session.java
##########
@@ -1541,7 +1541,7 @@ protected void sortTablet(Tablet tablet) {
int measurementSize = schema.getValueMeasurementIdList().size();
for (int j = 0; j < measurementSize; j++) {
tablet.values[columnIndex] =
- sortList(tablet.values[columnIndex], schema.getValueTSDataTypeList().get(i), index);
+ sortList(tablet.values[columnIndex], schema.getValueTSDataTypeList().get(j), index);
Review comment:
Added. Also add a complicated test case for mix MeasurementSchema and VectorMeasurementSchema in the same tablet : )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin commented on a change in pull request #2992: [To Vector] Fix Tablet.addValue()
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #2992:
URL: https://github.com/apache/iotdb/pull/2992#discussion_r611273841
##########
File path: session/src/main/java/org/apache/iotdb/session/Session.java
##########
@@ -1541,7 +1541,7 @@ protected void sortTablet(Tablet tablet) {
int measurementSize = schema.getValueMeasurementIdList().size();
for (int j = 0; j < measurementSize; j++) {
tablet.values[columnIndex] =
- sortList(tablet.values[columnIndex], schema.getValueTSDataTypeList().get(i), index);
+ sortList(tablet.values[columnIndex], schema.getValueTSDataTypeList().get(j), index);
Review comment:
add a test
##########
File path: example/session/src/main/java/org/apache/iotdb/VectorSessionExample.java
##########
@@ -215,18 +215,42 @@ private static void insertTabletWithAlignedTimeseries()
Tablet tablet = new Tablet(ROOT_SG1_D1, schemaList);
+ // Method 1 to add tablet data
+ long timestamp = System.currentTimeMillis();
+
+ for (long row = 0; row < 100; row++) {
+ int rowIndex = tablet.rowSize++;
+ tablet.addTimestamp(rowIndex, timestamp);
+ tablet.addValue(
+ schemaList.get(0).getValueMeasurementIdList().get(0), rowIndex, new Random().nextLong());
+ tablet.addValue(
+ schemaList.get(0).getValueMeasurementIdList().get(1), rowIndex, new Random().nextInt());
+
+ if (tablet.rowSize == tablet.getMaxRowNumber()) {
+ session.insertTablet(tablet, true);
+ tablet.reset();
+ }
+ timestamp++;
+ }
+
+ if (tablet.rowSize != 0) {
+ session.insertTablet(tablet);
+ tablet.reset();
+ }
+
+ // Method 2 to add tablet data
Review comment:
How about splitting these two methods into two functions, in case of users insert duplicated data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] HTHou commented on a change in pull request #2992: [To Vector] Fix Tablet.addValue()
Posted by GitBox <gi...@apache.org>.
HTHou commented on a change in pull request #2992:
URL: https://github.com/apache/iotdb/pull/2992#discussion_r611348197
##########
File path: tsfile/src/main/java/org/apache/iotdb/tsfile/write/record/Tablet.java
##########
@@ -121,13 +121,11 @@ public void addValue(String measurementId, int rowIndex, Object value) {
IMeasurementSchema measurementSchema = schemas.get(indexOfValue);
if (measurementSchema.getType().equals(TSDataType.VECTOR)) {
- for (int i = 0; i < measurementSchema.getValueMeasurementIdList().size(); i++) {
- TSDataType dataType = measurementSchema.getValueTSDataTypeList().get(i);
- addValueOfDataType(dataType, rowIndex, measurementIndex.get(measurementId), value);
- }
+ int indexInVector = measurementSchema.getMeasurementIdColumnIndex(measurementId);
+ TSDataType dataType = measurementSchema.getValueTSDataTypeList().get(indexInVector);
+ addValueOfDataType(dataType, rowIndex, indexOfValue + indexInVector, value);
Review comment:
We need to consider when `value == null`, init the `BitMaps` in tablet if needed, set an irrelevant value into the array, and mark this position of bitMap to represent this's a null value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] HTHou commented on a change in pull request #2992: [To Vector] Fix Tablet.addValue()
Posted by GitBox <gi...@apache.org>.
HTHou commented on a change in pull request #2992:
URL: https://github.com/apache/iotdb/pull/2992#discussion_r611348197
##########
File path: tsfile/src/main/java/org/apache/iotdb/tsfile/write/record/Tablet.java
##########
@@ -121,13 +121,11 @@ public void addValue(String measurementId, int rowIndex, Object value) {
IMeasurementSchema measurementSchema = schemas.get(indexOfValue);
if (measurementSchema.getType().equals(TSDataType.VECTOR)) {
- for (int i = 0; i < measurementSchema.getValueMeasurementIdList().size(); i++) {
- TSDataType dataType = measurementSchema.getValueTSDataTypeList().get(i);
- addValueOfDataType(dataType, rowIndex, measurementIndex.get(measurementId), value);
- }
+ int indexInVector = measurementSchema.getMeasurementIdColumnIndex(measurementId);
+ TSDataType dataType = measurementSchema.getValueTSDataTypeList().get(indexInVector);
+ addValueOfDataType(dataType, rowIndex, indexOfValue + indexInVector, value);
Review comment:
We need to consider when `value == null`, init the `BitMaps` in tablet if needed, set an irrelevant value in the array, and mark this position of bitMap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] samperson1997 commented on a change in pull request #2992: [To Vector] Fix Tablet.addValue()
Posted by GitBox <gi...@apache.org>.
samperson1997 commented on a change in pull request #2992:
URL: https://github.com/apache/iotdb/pull/2992#discussion_r612523076
##########
File path: example/session/src/main/java/org/apache/iotdb/VectorSessionExample.java
##########
@@ -215,18 +215,42 @@ private static void insertTabletWithAlignedTimeseries()
Tablet tablet = new Tablet(ROOT_SG1_D1, schemaList);
+ // Method 1 to add tablet data
+ long timestamp = System.currentTimeMillis();
+
+ for (long row = 0; row < 100; row++) {
+ int rowIndex = tablet.rowSize++;
+ tablet.addTimestamp(rowIndex, timestamp);
+ tablet.addValue(
+ schemaList.get(0).getValueMeasurementIdList().get(0), rowIndex, new Random().nextLong());
+ tablet.addValue(
+ schemaList.get(0).getValueMeasurementIdList().get(1), rowIndex, new Random().nextInt());
+
+ if (tablet.rowSize == tablet.getMaxRowNumber()) {
+ session.insertTablet(tablet, true);
+ tablet.reset();
+ }
+ timestamp++;
+ }
+
+ if (tablet.rowSize != 0) {
+ session.insertTablet(tablet);
+ tablet.reset();
+ }
+
+ // Method 2 to add tablet data
Review comment:
Splitted
##########
File path: example/session/src/main/java/org/apache/iotdb/VectorSessionExample.java
##########
@@ -215,18 +215,42 @@ private static void insertTabletWithAlignedTimeseries()
Tablet tablet = new Tablet(ROOT_SG1_D1, schemaList);
+ // Method 1 to add tablet data
+ long timestamp = System.currentTimeMillis();
+
+ for (long row = 0; row < 100; row++) {
+ int rowIndex = tablet.rowSize++;
+ tablet.addTimestamp(rowIndex, timestamp);
+ tablet.addValue(
+ schemaList.get(0).getValueMeasurementIdList().get(0), rowIndex, new Random().nextLong());
+ tablet.addValue(
+ schemaList.get(0).getValueMeasurementIdList().get(1), rowIndex, new Random().nextInt());
+
+ if (tablet.rowSize == tablet.getMaxRowNumber()) {
+ session.insertTablet(tablet, true);
+ tablet.reset();
+ }
+ timestamp++;
+ }
+
+ if (tablet.rowSize != 0) {
+ session.insertTablet(tablet);
+ tablet.reset();
+ }
+
+ // Method 2 to add tablet data
Review comment:
Splited
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] HTHou commented on a change in pull request #2992: [To Vector] Fix Tablet.addValue()
Posted by GitBox <gi...@apache.org>.
HTHou commented on a change in pull request #2992:
URL: https://github.com/apache/iotdb/pull/2992#discussion_r611630924
##########
File path: tsfile/src/main/java/org/apache/iotdb/tsfile/write/record/Tablet.java
##########
@@ -121,13 +121,11 @@ public void addValue(String measurementId, int rowIndex, Object value) {
IMeasurementSchema measurementSchema = schemas.get(indexOfValue);
if (measurementSchema.getType().equals(TSDataType.VECTOR)) {
- for (int i = 0; i < measurementSchema.getValueMeasurementIdList().size(); i++) {
- TSDataType dataType = measurementSchema.getValueTSDataTypeList().get(i);
- addValueOfDataType(dataType, rowIndex, measurementIndex.get(measurementId), value);
- }
+ int indexInVector = measurementSchema.getMeasurementIdColumnIndex(measurementId);
+ TSDataType dataType = measurementSchema.getValueTSDataTypeList().get(indexInVector);
+ addValueOfDataType(dataType, rowIndex, indexOfValue + indexInVector, value);
Review comment:
Fixed by myself...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] HTHou merged pull request #2992: [To Vector] Fix Tablet.addValue()
Posted by GitBox <gi...@apache.org>.
HTHou merged pull request #2992:
URL: https://github.com/apache/iotdb/pull/2992
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] HTHou commented on a change in pull request #2992: [To Vector] Fix Tablet.addValue()
Posted by GitBox <gi...@apache.org>.
HTHou commented on a change in pull request #2992:
URL: https://github.com/apache/iotdb/pull/2992#discussion_r611348197
##########
File path: tsfile/src/main/java/org/apache/iotdb/tsfile/write/record/Tablet.java
##########
@@ -121,13 +121,11 @@ public void addValue(String measurementId, int rowIndex, Object value) {
IMeasurementSchema measurementSchema = schemas.get(indexOfValue);
if (measurementSchema.getType().equals(TSDataType.VECTOR)) {
- for (int i = 0; i < measurementSchema.getValueMeasurementIdList().size(); i++) {
- TSDataType dataType = measurementSchema.getValueTSDataTypeList().get(i);
- addValueOfDataType(dataType, rowIndex, measurementIndex.get(measurementId), value);
- }
+ int indexInVector = measurementSchema.getMeasurementIdColumnIndex(measurementId);
+ TSDataType dataType = measurementSchema.getValueTSDataTypeList().get(indexInVector);
+ addValueOfDataType(dataType, rowIndex, indexOfValue + indexInVector, value);
Review comment:
We need to consider when `value == null`, init the `BitMaps` in tablet if needed, set an irrelevant value into the array, and mark this position of bitMap as the null value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org