You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by akashrn5 <gi...@git.apache.org> on 2018/04/21 10:20:17 UTC
[GitHub] carbondata pull request #2202: [wip]change datamap factory interface to chec...
GitHub user akashrn5 opened a pull request:
https://github.com/apache/carbondata/pull/2202
[wip]change datamap factory interface to check supported features
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/akashrn5/incubator-carbondata block_features_lucene
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2202.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2202
----
commit f5c0c49346f7ccdc7ec347417950a93d33ce56cb
Author: akashrn5 <ak...@...>
Date: 2018-04-21T10:00:55Z
change datamap factory interface to check supported features
----
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5482/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2202
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4492/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4165/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5505/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5331/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by akashrn5 <gi...@git.apache.org>.
Github user akashrn5 commented on the issue:
https://github.com/apache/carbondata/pull/2202
retest this please
---
[GitHub] carbondata pull request #2202: [CARBONDATA-2347][LUCENE]change datamap facto...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/2202
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4267/
---
[GitHub] carbondata pull request #2202: [wip]change datamap factory interface to chec...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2202#discussion_r183230585
--- Diff: core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java ---
@@ -2942,5 +2947,31 @@ public static String getBlockId(AbsoluteTableIdentifier identifier, String fileP
return blockId;
}
+ /**
+ * methods returns whether the feature is supported for the corresponding datamap or not
+ * @param carbonTable
+ * @param feature
+ * @return
+ */
+ public static boolean validateFeatureForDatamap(CarbonTable carbonTable, FeaturesList feature) {
+ boolean isSupported = false;
+ try {
+ List<TableDataMap> datamaps = DataMapStoreManager.getInstance().getAllDataMap(carbonTable);
+ if (!datamaps.isEmpty()) {
+ for (TableDataMap dataMap : datamaps) {
+ DataMapFactory factoryClass = DataMapStoreManager.getInstance()
+ .getDataMapFactoryClass(dataMap.getDataMapSchema());
+ isSupported = factoryClass.dataMapFeatureScope(feature);
--- End diff --
rename `dataMapFeatureScope` to `isOperationAllow`
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5353/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2202
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4627/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2202
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4476/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2202
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4608/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4146/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5452/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4287/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by akashrn5 <gi...@git.apache.org>.
Github user akashrn5 commented on the issue:
https://github.com/apache/carbondata/pull/2202
retest this please
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by akashrn5 <gi...@git.apache.org>.
Github user akashrn5 commented on the issue:
https://github.com/apache/carbondata/pull/2202
retest sdv please
---
[GitHub] carbondata pull request #2202: [wip]change datamap factory interface to chec...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2202#discussion_r183230695
--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableRenameCommand.scala ---
@@ -77,6 +78,11 @@ private[sql] case class CarbonAlterTableRenameCommand(
var oldCarbonTable: CarbonTable = null
oldCarbonTable = metastore.lookupRelation(Some(oldDatabaseName), oldTableName)(sparkSession)
.asInstanceOf[CarbonRelation].carbonTable
+
+ if (CarbonUtil.validateFeatureForDatamap(oldCarbonTable, FeaturesList.ALTER_RENAME)) {
--- End diff --
After you handle previous comments, I think this line will become:
` if (oldCarbonTable.canAccept(TableOperation.ALTER_RENAME))`
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2202
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4574/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4340/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5423/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4311/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2202
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4538/
---
[GitHub] carbondata issue #2202: [wip]change datamap factory interface to check suppo...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5277/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:
https://github.com/apache/carbondata/pull/2202
LGTM
---
[GitHub] carbondata issue #2202: [wip]change datamap factory interface to check suppo...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4098/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5400/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4177/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by akashrn5 <gi...@git.apache.org>.
Github user akashrn5 commented on the issue:
https://github.com/apache/carbondata/pull/2202
retest this please
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4233/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5434/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4359/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4315/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2202
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4559/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2202
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4483/
---
[GitHub] carbondata pull request #2202: [wip]change datamap factory interface to chec...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2202#discussion_r183230388
--- Diff: core/src/main/java/org/apache/carbondata/core/features/FeaturesList.java ---
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.core.features;
+
+public enum FeaturesList {
--- End diff --
change to `TableCommand`
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5321/
---
[GitHub] carbondata pull request #2202: [wip]change datamap factory interface to chec...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2202#discussion_r183230516
--- Diff: core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java ---
@@ -2942,5 +2947,31 @@ public static String getBlockId(AbsoluteTableIdentifier identifier, String fileP
return blockId;
}
+ /**
+ * methods returns whether the feature is supported for the corresponding datamap or not
+ * @param carbonTable
+ * @param feature
+ * @return
+ */
+ public static boolean validateFeatureForDatamap(CarbonTable carbonTable, FeaturesList feature) {
--- End diff --
move this function to CarbonTable
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4193/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4273/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5368/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2202
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4539/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5478/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2202
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4578/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2202
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4498/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4257/
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5523/
---
[GitHub] carbondata pull request #2202: [wip]change datamap factory interface to chec...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2202#discussion_r183230534
--- Diff: core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java ---
@@ -2942,5 +2947,31 @@ public static String getBlockId(AbsoluteTableIdentifier identifier, String fileP
return blockId;
}
+ /**
+ * methods returns whether the feature is supported for the corresponding datamap or not
+ * @param carbonTable
+ * @param feature
+ * @return
+ */
+ public static boolean validateFeatureForDatamap(CarbonTable carbonTable, FeaturesList feature) {
+ boolean isSupported = false;
+ try {
+ List<TableDataMap> datamaps = DataMapStoreManager.getInstance().getAllDataMap(carbonTable);
+ if (!datamaps.isEmpty()) {
+ for (TableDataMap dataMap : datamaps) {
+ DataMapFactory factoryClass = DataMapStoreManager.getInstance()
--- End diff --
I suggest to change `DataMapFactory` class name to `IndexDataMap`
---
[GitHub] carbondata issue #2202: [CARBONDATA-2347][LUCENE]change datamap factory inte...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2202
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4152/
---