You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/08/05 07:41:12 UTC
[GitHub] [incubator-doris] Youngwb opened a new pull request #4256: Support sequence column for UNIQUE_KEYS Table
Youngwb opened a new pull request #4256:
URL: https://github.com/apache/incubator-doris/pull/4256
For #3930
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #4256: Support sequence column for UNIQUE_KEYS Table
Posted by GitBox <gi...@apache.org>.
yangzhg commented on a change in pull request #4256:
URL: https://github.com/apache/incubator-doris/pull/4256#discussion_r467663294
##########
File path: be/src/olap/tablet_schema.h
##########
@@ -111,6 +111,8 @@ class TabletSchema {
inline double bloom_filter_fpp() const { return _bf_fpp; }
inline bool is_in_memory() const { return _is_in_memory; }
inline void set_is_in_memory(bool is_in_memory) { _is_in_memory = is_in_memory; }
+ inline bool has_sequence_col() const { return _has_sequence_col; }
Review comment:
how about checking sequence_col_idx != -1 instead of _has_sequence_col
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] Youngwb commented on a change in pull request #4256: Support sequence column for UNIQUE_KEYS Table
Posted by GitBox <gi...@apache.org>.
Youngwb commented on a change in pull request #4256:
URL: https://github.com/apache/incubator-doris/pull/4256#discussion_r478897131
##########
File path: fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
##########
@@ -796,6 +801,43 @@ public void setBloomFilterInfo(Set<String> bfColumns, double bfFpp) {
this.bfFpp = bfFpp;
}
+ public void setSequenceInfo(Type type) {
+ this.hasSequenceCol = true;
+ this.sequenceType = type;
+
+ // sequence column is value column with MAX aggregate type
+ // this makes it easy to compare and update the sequence column
+ Column sequenceCol = new Column(Column.SEQUENCE_COL, type, false, AggregateType.MAX, false, null, "", false);
Review comment:
I have updated, sequence column use REPLACE
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #4256: Support sequence column for UNIQUE_KEYS Table
Posted by GitBox <gi...@apache.org>.
yangzhg commented on a change in pull request #4256:
URL: https://github.com/apache/incubator-doris/pull/4256#discussion_r467671444
##########
File path: fe/fe-core/src/main/java/org/apache/doris/alter/MaterializedViewHandler.java
##########
@@ -537,6 +538,13 @@ private RollupJobV2 createMaterializedViewJob(String mvName, String baseIndexNam
}
}
}
+ if (KeysType.UNIQUE_KEYS == keysType && olapTable.hasSequence()) {
+ if (meetValue) {
+ // add the sequence column
+ rollupSchema.add(keysNumOfRollup, new Column(olapTable.sequenceColName, olapTable.getSequenceType(),
+ false, AggregateType.MAX, null, ""));
Review comment:
why do not check if sequenceColName ia alread exist in rollupSchema
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #4256: Support sequence column for UNIQUE_KEYS Table
Posted by GitBox <gi...@apache.org>.
yangzhg merged pull request #4256:
URL: https://github.com/apache/incubator-doris/pull/4256
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #4256: Support sequence column for UNIQUE_KEYS Table
Posted by GitBox <gi...@apache.org>.
yangzhg commented on a change in pull request #4256:
URL: https://github.com/apache/incubator-doris/pull/4256#discussion_r467766872
##########
File path: fe/fe-core/src/main/java/org/apache/doris/common/FeMetaVersion.java
##########
@@ -190,6 +190,8 @@
// force drop db, force drop table, force drop partition
// make force drop operation do not recycle meta
public static final int VERSION_89 = 89;
+ // sequence column for unique_keys
+ public static final int VERSION_90 = 90;
// note: when increment meta version, should assign the latest version to VERSION_CURRENT
- public static final int VERSION_CURRENT = VERSION_89;
+ public static final int VERSION_CURRENT = VERSION_90;
Review comment:
not using ？
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] Youngwb commented on a change in pull request #4256: Support sequence column for UNIQUE_KEYS Table
Posted by GitBox <gi...@apache.org>.
Youngwb commented on a change in pull request #4256:
URL: https://github.com/apache/incubator-doris/pull/4256#discussion_r467789336
##########
File path: be/src/olap/tablet_schema.h
##########
@@ -111,6 +111,8 @@ class TabletSchema {
inline double bloom_filter_fpp() const { return _bf_fpp; }
inline bool is_in_memory() const { return _is_in_memory; }
inline void set_is_in_memory(bool is_in_memory) { _is_in_memory = is_in_memory; }
+ inline bool has_sequence_col() const { return _has_sequence_col; }
Review comment:
modified
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] Youngwb commented on a change in pull request #4256: Support sequence column for UNIQUE_KEYS Table
Posted by GitBox <gi...@apache.org>.
Youngwb commented on a change in pull request #4256:
URL: https://github.com/apache/incubator-doris/pull/4256#discussion_r467787309
##########
File path: fe/fe-core/src/main/java/org/apache/doris/common/FeMetaVersion.java
##########
@@ -190,6 +190,8 @@
// force drop db, force drop table, force drop partition
// make force drop operation do not recycle meta
public static final int VERSION_89 = 89;
+ // sequence column for unique_keys
+ public static final int VERSION_90 = 90;
// note: when increment meta version, should assign the latest version to VERSION_CURRENT
- public static final int VERSION_CURRENT = VERSION_89;
+ public static final int VERSION_CURRENT = VERSION_90;
Review comment:
mistake, I have already correct it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] Youngwb commented on a change in pull request #4256: Support sequence column for UNIQUE_KEYS Table
Posted by GitBox <gi...@apache.org>.
Youngwb commented on a change in pull request #4256:
URL: https://github.com/apache/incubator-doris/pull/4256#discussion_r467788869
##########
File path: fe/fe-core/src/main/java/org/apache/doris/alter/MaterializedViewHandler.java
##########
@@ -537,6 +538,13 @@ private RollupJobV2 createMaterializedViewJob(String mvName, String baseIndexNam
}
}
}
+ if (KeysType.UNIQUE_KEYS == keysType && olapTable.hasSequence()) {
+ if (meetValue) {
+ // add the sequence column
+ rollupSchema.add(keysNumOfRollup, new Column(olapTable.sequenceColName, olapTable.getSequenceType(),
+ false, AggregateType.MAX, null, ""));
Review comment:
add check Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] Youngwb commented on a change in pull request #4256: Support sequence column for UNIQUE_KEYS Table
Posted by GitBox <gi...@apache.org>.
Youngwb commented on a change in pull request #4256:
URL: https://github.com/apache/incubator-doris/pull/4256#discussion_r478801657
##########
File path: be/test/olap/test_data/header.txt
##########
@@ -50,7 +50,8 @@
"compress_kind": "COMPRESS_LZ4",
"next_column_unique_id": 3,
"is_in_memory": false,
- "delete_sign_idx": -1
+ "delete_sign_idx": -1,
+ "has_sequence_col": false
Review comment:
fix
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] Youngwb commented on a change in pull request #4256: Support sequence column for UNIQUE_KEYS Table
Posted by GitBox <gi...@apache.org>.
Youngwb commented on a change in pull request #4256:
URL: https://github.com/apache/incubator-doris/pull/4256#discussion_r478897110
##########
File path: fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
##########
@@ -796,6 +801,43 @@ public void setBloomFilterInfo(Set<String> bfColumns, double bfFpp) {
this.bfFpp = bfFpp;
}
+ public void setSequenceInfo(Type type) {
+ this.hasSequenceCol = true;
+ this.sequenceType = type;
+
+ // sequence column is value column with MAX aggregate type
+ // this makes it easy to compare and update the sequence column
+ Column sequenceCol = new Column(Column.SEQUENCE_COL, type, false, AggregateType.MAX, false, null, "", false);
Review comment:
I have updated, sequence column use REPLACE
##########
File path: fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
##########
@@ -796,6 +801,43 @@ public void setBloomFilterInfo(Set<String> bfColumns, double bfFpp) {
this.bfFpp = bfFpp;
}
+ public void setSequenceInfo(Type type) {
+ this.hasSequenceCol = true;
+ this.sequenceType = type;
+
+ // sequence column is value column with MAX aggregate type
+ // this makes it easy to compare and update the sequence column
+ Column sequenceCol = new Column(Column.SEQUENCE_COL, type, false, AggregateType.MAX, false, null, "", false);
Review comment:
I have updated, sequence column use REPLACE
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #4256: Support sequence column for UNIQUE_KEYS Table
Posted by GitBox <gi...@apache.org>.
yangzhg commented on a change in pull request #4256:
URL: https://github.com/apache/incubator-doris/pull/4256#discussion_r478799921
##########
File path: be/test/olap/test_data/header.txt
##########
@@ -50,7 +50,8 @@
"compress_kind": "COMPRESS_LZ4",
"next_column_unique_id": 3,
"is_in_memory": false,
- "delete_sign_idx": -1
+ "delete_sign_idx": -1,
+ "has_sequence_col": false
Review comment:
indent？
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #4256: Support sequence column for UNIQUE_KEYS Table
Posted by GitBox <gi...@apache.org>.
yangzhg commented on a change in pull request #4256:
URL: https://github.com/apache/incubator-doris/pull/4256#discussion_r478808980
##########
File path: fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
##########
@@ -796,6 +801,43 @@ public void setBloomFilterInfo(Set<String> bfColumns, double bfFpp) {
this.bfFpp = bfFpp;
}
+ public void setSequenceInfo(Type type) {
+ this.hasSequenceCol = true;
+ this.sequenceType = type;
+
+ // sequence column is value column with MAX aggregate type
+ // this makes it easy to compare and update the sequence column
+ Column sequenceCol = new Column(Column.SEQUENCE_COL, type, false, AggregateType.MAX, false, null, "", false);
Review comment:
add AggregateType.MAX to unique table ??
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org