You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by GitBox <gi...@apache.org> on 2020/03/04 09:49:21 UTC
[GitHub] [carbondata] akashrn5 opened a new pull request #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
akashrn5 opened a new pull request #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656
### Why is this PR needed?
### What changes were proposed in this PR?
### Does this PR introduce any user interface change?
- No
- Yes. (please explain the change and update document)
### Is any new testcase added?
- No
- Yes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on a change in pull request #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on a change in pull request #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#discussion_r388221619
##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/load/CarbonInternalLoaderUtil.java
##########
@@ -308,17 +308,17 @@ public static boolean updateLoadMetadataWithMergeStatus(CarbonTable indexCarbonT
*/
public static boolean checkMainTableSegEqualToSISeg(String carbonTablePath,
String indexTablePath) {
- List<String> mainList =
- getListOfValidSlices(SegmentStatusManager.readLoadMetadata(carbonTablePath));
+ List<String> mainListCarbonSegments =
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] kunal642 commented on issue #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
kunal642 commented on issue #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#issuecomment-601540376
LGTM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#issuecomment-594470617
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2314/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#issuecomment-599546138
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2480/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#issuecomment-599529077
Build Failed with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/773/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] Indhumathi27 commented on a change in pull request
#3656: [CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on a change in pull request #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#discussion_r388076346
##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/load/CarbonInternalLoaderUtil.java
##########
@@ -308,17 +308,17 @@ public static boolean updateLoadMetadataWithMergeStatus(CarbonTable indexCarbonT
*/
public static boolean checkMainTableSegEqualToSISeg(String carbonTablePath,
String indexTablePath) {
- List<String> mainList =
- getListOfValidSlices(SegmentStatusManager.readLoadMetadata(carbonTablePath));
+ List<String> mainListCarbonSegments =
Review comment:
```suggestion
List<String> mainTableSegmentsList=
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on a change in pull request #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on a change in pull request #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#discussion_r388221782
##########
File path: core/src/main/java/org/apache/carbondata/core/datamap/DataMapFilter.java
##########
@@ -100,6 +107,19 @@ public void setTable(CarbonTable table) {
this.table = table;
}
+ private void initializeExternalSegmentFilter() {
+ if ((expression instanceof AndExpression) && (((AndExpression) expression)
+ .getRight() instanceof InExpression) && (expression.getChildren().get(1).getChildren()
+ .get(0) instanceof ColumnExpression) && (((ColumnExpression) expression.getChildren().get(1)
+ .getChildren().get(0))).getColumnName().equalsIgnoreCase("positionid")) {
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] Indhumathi27 commented on a change in pull request
#3656: [CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on a change in pull request #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#discussion_r388077137
##########
File path: core/src/main/java/org/apache/carbondata/core/datamap/DataMapFilter.java
##########
@@ -100,6 +107,19 @@ public void setTable(CarbonTable table) {
this.table = table;
}
+ private void initializeExternalSegmentFilter() {
+ if ((expression instanceof AndExpression) && (((AndExpression) expression)
+ .getRight() instanceof InExpression) && (expression.getChildren().get(1).getChildren()
+ .get(0) instanceof ColumnExpression) && (((ColumnExpression) expression.getChildren().get(1)
+ .getChildren().get(0))).getColumnName().equalsIgnoreCase("positionid")) {
Review comment:
Can use **CarbonCommonConstants.CARBON_IMPLICIT_COLUMN_POSITIONID** instead of `positionid`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on issue #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on issue #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#issuecomment-601538241
@kunal642 fixed all test cases, please review and merge
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] Indhumathi27 commented on a change in pull request
#3656: [CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on a change in pull request #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#discussion_r388077137
##########
File path: core/src/main/java/org/apache/carbondata/core/datamap/DataMapFilter.java
##########
@@ -100,6 +107,19 @@ public void setTable(CarbonTable table) {
this.table = table;
}
+ private void initializeExternalSegmentFilter() {
+ if ((expression instanceof AndExpression) && (((AndExpression) expression)
+ .getRight() instanceof InExpression) && (expression.getChildren().get(1).getChildren()
+ .get(0) instanceof ColumnExpression) && (((ColumnExpression) expression.getChildren().get(1)
+ .getChildren().get(0))).getColumnName().equalsIgnoreCase("positionid")) {
Review comment:
Can use **CarbonCommonConstants.POSITION_ID** instead of `positionid`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] asfgit closed pull request #3656: [CARBONDATA-3680]Fix
add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] Indhumathi27 commented on a change in pull request
#3656: [CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on a change in pull request #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#discussion_r388077137
##########
File path: core/src/main/java/org/apache/carbondata/core/datamap/DataMapFilter.java
##########
@@ -100,6 +107,19 @@ public void setTable(CarbonTable table) {
this.table = table;
}
+ private void initializeExternalSegmentFilter() {
+ if ((expression instanceof AndExpression) && (((AndExpression) expression)
+ .getRight() instanceof InExpression) && (expression.getChildren().get(1).getChildren()
+ .get(0) instanceof ColumnExpression) && (((ColumnExpression) expression.getChildren().get(1)
+ .getChildren().get(0))).getColumnName().equalsIgnoreCase("positionid")) {
Review comment:
Can use CarbonCommonConstants.CARBON_IMPLICIT_COLUMN_POSITIONID instead of `positionid`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#issuecomment-601360576
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2511/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on a change in pull request #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on a change in pull request #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#discussion_r388221691
##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonAddLoadCommand.scala
##########
@@ -277,6 +277,7 @@ case class CarbonAddLoadCommand(
operationContext.setProperty(
carbonTable.getTableUniqueName + "_Segment",
model.getSegmentId)
+ operationContext.setProperty("isAddLoad", "true")
Review comment:
added
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#issuecomment-595208940
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2343/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] Indhumathi27 commented on a change in pull request
#3656: [CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on a change in pull request #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#discussion_r388076650
##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonAddLoadCommand.scala
##########
@@ -277,6 +277,7 @@ case class CarbonAddLoadCommand(
operationContext.setProperty(
carbonTable.getTableUniqueName + "_Segment",
model.getSegmentId)
+ operationContext.setProperty("isAddLoad", "true")
Review comment:
Please add comments for the usage of `isAddLoad`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#issuecomment-601358432
Build Success with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/806/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#issuecomment-594466946
Build Success with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/607/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] kunal642 commented on issue #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
kunal642 commented on issue #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#issuecomment-601001508
@akashrn5 Please fix the CI failures
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3656:
[CARBONDATA-3680]Fix add segment issue with SI table
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3656: [CARBONDATA-3680]Fix add segment issue with SI table
URL: https://github.com/apache/carbondata/pull/3656#issuecomment-595207986
Build Success with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/636/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services